-
Large Language Models as Planning Domain Generators
Authors:
James Oswald,
Kavitha Srinivas,
Harsha Kokel,
Junkyu Lee,
Michael Katz,
Shirin Sohrabi
Abstract:
Develo** domain models is one of the few remaining places that require manual human labor in AI planning. Thus, in order to make planning more accessible, it is desirable to automate the process of domain model generation. To this end, we investigate if large language models (LLMs) can be used to generate planning domain models from simple textual descriptions. Specifically, we introduce a frame…
▽ More
Develo** domain models is one of the few remaining places that require manual human labor in AI planning. Thus, in order to make planning more accessible, it is desirable to automate the process of domain model generation. To this end, we investigate if large language models (LLMs) can be used to generate planning domain models from simple textual descriptions. Specifically, we introduce a framework for automated evaluation of LLM-generated domains by comparing the sets of plans for domain instances. Finally, we perform an empirical analysis of 7 large language models, including coding and chat models across 9 different planning domains, and under three classes of natural language domain descriptions. Our results indicate that LLMs, particularly those with high parameter counts, exhibit a moderate level of proficiency in generating correct planning domains from natural language descriptions. Our code is available at https://github.com/IBM/NL2PDDL.
△ Less
Submitted 2 April, 2024;
originally announced May 2024.
-
Thought of Search: Planning with Language Models Through The Lens of Efficiency
Authors:
Michael Katz,
Harsha Kokel,
Kavitha Srinivas,
Shirin Sohrabi
Abstract:
Among the most important properties of algorithms investigated in computer science are soundness, completeness, and complexity. These properties, however, are rarely analyzed for the vast collection of recently proposed methods for planning with large language models. In this work, we alleviate this gap. We analyse these properties of using LLMs for planning and highlight that recent trends abando…
▽ More
Among the most important properties of algorithms investigated in computer science are soundness, completeness, and complexity. These properties, however, are rarely analyzed for the vast collection of recently proposed methods for planning with large language models. In this work, we alleviate this gap. We analyse these properties of using LLMs for planning and highlight that recent trends abandon both soundness and completeness for the sake of inefficiency. We propose a significantly more efficient approach that can, at the same time, maintain both soundness and completeness. We exemplify on four representative search problems, comparing to the LLM-based solutions from the literature that attempt to solve these problems. We show that by using LLMs to produce the code for the search components we can solve the entire datasets with 100\% accuracy with only a few calls to the LLM. We argue for a responsible use of compute resources; urging research community to investigate sound and complete LLM-based approaches that uphold efficiency.
△ Less
Submitted 21 May, 2024; v1 submitted 17 April, 2024;
originally announced April 2024.
-
Some Orders Are Important: Partially Preserving Orders in Top-Quality Planning
Authors:
Michael Katz,
Junkyu Lee,
Jungkoo Kang,
Shirin Sohrabi
Abstract:
The ability to generate multiple plans is central to using planning in real-life applications. Top-quality planners generate sets of such top-cost plans, allowing flexibility in determining equivalent ones. In terms of the order between actions in a plan, the literature only considers two extremes -- either all orders are important, making each plan unique, or all orders are unimportant, treating…
▽ More
The ability to generate multiple plans is central to using planning in real-life applications. Top-quality planners generate sets of such top-cost plans, allowing flexibility in determining equivalent ones. In terms of the order between actions in a plan, the literature only considers two extremes -- either all orders are important, making each plan unique, or all orders are unimportant, treating two plans differing only in the order of actions as equivalent. To allow flexibility in selecting important orders, we propose specifying a subset of actions the orders between which are important, interpolating between the top-quality and unordered top-quality planning problems. We explore the ways of adapting partial order reduction search pruning techniques to address this new computational problem and present experimental evaluations demonstrating the benefits of exploiting such techniques in this setting.
△ Less
Submitted 1 April, 2024;
originally announced April 2024.
-
Unifying and Certifying Top-Quality Planning
Authors:
Michael Katz,
Junkyu Lee,
Shirin Sohrabi
Abstract:
The growing utilization of planning tools in practical scenarios has sparked an interest in generating multiple high-quality plans. Consequently, a range of computational problems under the general umbrella of top-quality planning were introduced over a short time period, each with its own definition. In this work, we show that the existing definitions can be unified into one, based on a dominance…
▽ More
The growing utilization of planning tools in practical scenarios has sparked an interest in generating multiple high-quality plans. Consequently, a range of computational problems under the general umbrella of top-quality planning were introduced over a short time period, each with its own definition. In this work, we show that the existing definitions can be unified into one, based on a dominance relation. The different computational problems, therefore, simply correspond to different dominance relations. Given the unified definition, we can now certify the top-quality of the solutions, leveraging existing certification of unsolvability and optimality. We show that task transformations found in the existing literature can be employed for the efficient certification of various top-quality planning problems and propose a novel transformation to efficiently certify loopless top-quality planning.
△ Less
Submitted 5 March, 2024;
originally announced March 2024.
-
Hierarchical Reinforcement Learning with AI Planning Models
Authors:
Junkyu Lee,
Michael Katz,
Don Joven Agravante,
Miao Liu,
Geraud Nangue Tasse,
Tim Klinger,
Shirin Sohrabi
Abstract:
Two common approaches to sequential decision-making are AI planning (AIP) and reinforcement learning (RL). Each has strengths and weaknesses. AIP is interpretable, easy to integrate with symbolic knowledge, and often efficient, but requires an up-front logical domain specification and is sensitive to noise; RL only requires specification of rewards and is robust to noise but is sample inefficient…
▽ More
Two common approaches to sequential decision-making are AI planning (AIP) and reinforcement learning (RL). Each has strengths and weaknesses. AIP is interpretable, easy to integrate with symbolic knowledge, and often efficient, but requires an up-front logical domain specification and is sensitive to noise; RL only requires specification of rewards and is robust to noise but is sample inefficient and not easily supplied with external knowledge. We propose an integrative approach that combines high-level planning with RL, retaining interpretability, transfer, and efficiency, while allowing for robust learning of the lower-level plan actions. Our approach defines options in hierarchical reinforcement learning (HRL) from AIP operators by establishing a correspondence between the state transition model of AI planning problem and the abstract state transition system of a Markov Decision Process (MDP). Options are learned by adding intrinsic rewards to encourage consistency between the MDP and AIP transition models. We demonstrate the benefit of our integrated approach by comparing the performance of RL and HRL algorithms in both MiniGrid and N-rooms environments, showing the advantage of our method over the existing ones.
△ Less
Submitted 28 September, 2022; v1 submitted 1 March, 2022;
originally announced March 2022.
-
Reinforcement Learning for Classical Planning: Viewing Heuristics as Dense Reward Generators
Authors:
Clement Gehring,
Masataro Asai,
Rohan Chitnis,
Tom Silver,
Leslie Pack Kaelbling,
Shirin Sohrabi,
Michael Katz
Abstract:
Recent advances in reinforcement learning (RL) have led to a growing interest in applying RL to classical planning domains or applying classical planning methods to some complex RL domains. However, the long-horizon goal-based problems found in classical planning lead to sparse rewards for RL, making direct application inefficient. In this paper, we propose to leverage domain-independent heuristic…
▽ More
Recent advances in reinforcement learning (RL) have led to a growing interest in applying RL to classical planning domains or applying classical planning methods to some complex RL domains. However, the long-horizon goal-based problems found in classical planning lead to sparse rewards for RL, making direct application inefficient. In this paper, we propose to leverage domain-independent heuristic functions commonly used in the classical planning literature to improve the sample efficiency of RL. These classical heuristics act as dense reward generators to alleviate the sparse-rewards issue and enable our RL agent to learn domain-specific value functions as residuals on these heuristics, making learning easier. Correct application of this technique requires consolidating the discounted metric used in RL and the non-discounted metric used in heuristics. We implement the value functions using Neural Logic Machines, a neural network architecture designed for grounded first-order logic inputs. We demonstrate on several classical planning domains that using classical heuristics for RL allows for good sample efficiency compared to sparse-reward RL. We further show that our learned value functions generalize to novel problem instances in the same domain.
△ Less
Submitted 7 March, 2022; v1 submitted 29 September, 2021;
originally announced September 2021.
-
Knowledge Engineering for Planning-Based Hypothesis Generation
Authors:
Shirin Sohrabi,
Octavian Udrea,
Anton V. Riabov
Abstract:
In this paper, we address the knowledge engineering problems for hypothesis generation motivated by applications that require timely exploration of hypotheses under unreliable observations. We looked at two applications: malware detection and intensive care delivery. In intensive care, the goal is to generate plausible hypotheses about the condition of the patient from clinical observations and fu…
▽ More
In this paper, we address the knowledge engineering problems for hypothesis generation motivated by applications that require timely exploration of hypotheses under unreliable observations. We looked at two applications: malware detection and intensive care delivery. In intensive care, the goal is to generate plausible hypotheses about the condition of the patient from clinical observations and further refine these hypotheses to create a recovery plan for the patient. Similarly, preventing malware spread within a corporate network involves generating hypotheses from network traffic data and selecting preventive actions. To this end, building on the already established characterization and use of AI planning for similar problems, we propose use of planning for the hypothesis generation problem. However, to deal with uncertainty, incomplete model description and unreliable observations, we need to use a planner capable of generating multiple high-quality plans. To capture the model description we propose a language called LTS++ and a web-based tool that enables the specification of the LTS++ model and a set of observations. We also proposed a 9-step process that helps provide guidance to the domain expert in specifying the LTS++ model. The hypotheses are then generated by running a planner on the translated LTS++ model and the provided trace. The hypotheses can be visualized and shown to the analyst or can be further investigated automatically.
△ Less
Submitted 27 August, 2014;
originally announced August 2014.
-
Cohen-Macaulay and Gorenstein properties under the amalgamated construction
Authors:
P. Sahandi,
N. Shirmohammadi,
S. Sohrabi
Abstract:
Let $A$ and $B$ be commutative rings with unity, $f:A\to B$ a ring homomorphism and $J$ an ideal of $B$. Then the subring $A\bowtie^fJ:=\{(a,f(a)+j)|a\in A$ and $j\in J\}$ of $A\times B$ is called the amalgamation of $A$ with $B$ along with $J$ with respect to $f$. In this paper, among other things, we investigate the Cohen-Macaulay and (quasi-)Gorenstein properties on the ring $A\bowtie^fJ$.
Let $A$ and $B$ be commutative rings with unity, $f:A\to B$ a ring homomorphism and $J$ an ideal of $B$. Then the subring $A\bowtie^fJ:=\{(a,f(a)+j)|a\in A$ and $j\in J\}$ of $A\times B$ is called the amalgamation of $A$ with $B$ along with $J$ with respect to $f$. In this paper, among other things, we investigate the Cohen-Macaulay and (quasi-)Gorenstein properties on the ring $A\bowtie^fJ$.
△ Less
Submitted 8 December, 2014; v1 submitted 9 June, 2014;
originally announced June 2014.
-
On Planning with Preferences in HTN
Authors:
Shirin Sohrabi,
Sheila A. McIlraith
Abstract:
In this paper, we address the problem of generating preferred plans by combining the procedural control knowledge specified by Hierarchical Task Networks (HTNs) with rich qualitative user preferences. The outcome of our work is a language for specifyin user preferences, tailored to HTN planning, together with a provably optimal preference-based planner, HTNPLAN, that is implemented as an extensi…
▽ More
In this paper, we address the problem of generating preferred plans by combining the procedural control knowledge specified by Hierarchical Task Networks (HTNs) with rich qualitative user preferences. The outcome of our work is a language for specifyin user preferences, tailored to HTN planning, together with a provably optimal preference-based planner, HTNPLAN, that is implemented as an extension of SHOP2. To compute preferred plans, we propose an approach based on forward-chaining heuristic search. Our heuristic uses an admissible evaluation function measuring the satisfaction of preferences over partial plans. Our empirical evaluation demonstrates the effectiveness of our HTNPLAN heuristics. We prove our approach sound and optimal with respect to the plans it generates by appealing to a situation calculus semantics of our preference language and of HTN planning. While our implementation builds on SHOP2, the language and techniques proposed here are relevant to a broad range of HTN planners.
△ Less
Submitted 3 September, 2009;
originally announced September 2009.
