[논문 리뷰] On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)

Vishal Pallagani, Kaushik Roy외 8명|arXiv (Cornell University)|2024. 01. 04.

AI-based Problem Solving and Planning참고 문헌 129인용 수 7

한 줄 요약

이 위치 논문은 126개의 연구를 조사하여 LLMs를 APS에 활용하는 것을 다루고, 전통적 플래너와의 신경-기호 통합을 주장합니다.

ABSTRACT

Automated Planning and Scheduling is among the growing areas in Artificial Intelligence (AI) where mention of LLMs has gained popularity. Based on a comprehensive review of 126 papers, this paper investigates eight categories based on the unique applications of LLMs in addressing various aspects of planning problems: language translation, plan generation, model construction, multi-agent planning, interactive planning, heuristics optimization, tool integration, and brain-inspired planning. For each category, we articulate the issues considered and existing gaps. A critical insight resulting from our review is that the true potential of LLMs unfolds when they are integrated with traditional symbolic planners, pointing towards a promising neuro-symbolic approach. This approach effectively combines the generative aspects of LLMs with the precision of classical planning methods. By synthesizing insights from existing literature, we underline the potential of this integration to address complex planning challenges. Our goal is to encourage the ICAPS community to recognize the complementary strengths of LLMs and symbolic planners, advocating for a direction in automated planning that leverages these synergistic capabilities to develop more advanced and intelligent planning systems.

연구 동기 및 목표

• Motivate integrating LLMs with classical APS to address flexibility and contextual adaptability gaps.
• Categorize LLM applications in APS into eight distinct areas for structured insight.
• Identify gaps and limitations in current LLM-based APS approaches to guide future research.
• Promote a neuro-symbolic AI paradigm as a core direction for advancing APS capabilities.

제안 방법

• Conduct a comprehensive literature review of 126 papers on LLMs in APS.
• Classify papers into eight categories: Language Translation, Plan Generation, Model Construction, Multi-agent Planning, Interactive Planning, Heuristics Optimization, Tool Integration, Brain-Inspired Planning.
• Provide a qualitative synthesis of strengths, gaps, and future opportunities per category.
• Extract and synthesize insights to advocate for neuro-symbolic integration with symbolic planners.
• Use a manual categorization process with cross-author review to ensure coherent taxonomy and coverage.

Figure 1: Radar chart showcasing the relative performance of six language models (GPT-4, Claude-v1, GPT-3.5-turbo, Vicuna-13B, Alpaca-13B, LLama-13B) across key domains: Writing, Roleplay, Reasoning, Math, Coding, Extraction, STEM, and Humanities from Zheng et al. ( 2023a ) .

실험 결과

연구 질문

• RQ1 What are the eight categories of LLM applications in APS and what issues/gaps characterize each?
• RQ2 To what extent do LLMs complement or replace traditional symbolic planners across these categories?
• RQ3 What is the evidence for neuro-symbolic integration as a pathway to improved planning systems?

주요 결과

• Eight-category taxonomy established for LLMs in APS, with Plan Generation being the most explored (53 papers) and Brain-Inspired Planning least represented (5 papers).
• LLMs excel at Language Translation but face limits in plan optimality, completeness, and generalization in Plan Generation.
• Neuro-symbolic approaches are identified as a promising direction to combine LLMs’ language capabilities with the precision of symbolic planners.
• Interactive Planning and Tool Integration show strong potential for adaptability and cross-system coordination, despite challenges like tool over-reliance and grounding.
• Gaps include grounding, grounding-of-affordances, standardization of inter-agent communication, and the need for world models to improve low-level reasoning.
• The study argues for integrating LLMs with symbolic planners to achieve dynamic, context-aware, and scalable APS.

Figure 2: Of the 126 papers surveyed in this study, 55 were accepted by peer-reviewed conferences. This chart illustrates the distribution of these papers across various conferences in the fields of LLMs and APS, highlighting the primary forums for scholarly contributions in these areas.