[论文解读] A Survey on Large Language Model based Autonomous Agents

Lei Wang, Chen Ma+11 more|arXiv (Cornell University)|Aug 22, 2023

Topic Modeling被引用 113

一句话总结

本综述汇聚并分析基于 LLM 的自治代理，提出一个统一架构（画像、记忆、计划、行动），并评审它们的构建、应用及评估，提供分类体系和未来方向。

ABSTRACT

Autonomous agents have long been a prominent research focus in both academic and industry communities. Previous research in this field often focuses on training agents with limited knowledge within isolated environments, which diverges significantly from human learning processes, and thus makes the agents hard to achieve human-like decisions. Recently, through the acquisition of vast amounts of web knowledge, large language models (LLMs) have demonstrated remarkable potential in achieving human-level intelligence. This has sparked an upsurge in studies investigating LLM-based autonomous agents. In this paper, we present a comprehensive survey of these studies, delivering a systematic review of the field of LLM-based autonomous agents from a holistic perspective. More specifically, we first discuss the construction of LLM-based autonomous agents, for which we propose a unified framework that encompasses a majority of the previous work. Then, we present a comprehensive overview of the diverse applications of LLM-based autonomous agents in the fields of social science, natural science, and engineering. Finally, we delve into the evaluation strategies commonly used for LLM-based autonomous agents. Based on the previous studies, we also present several challenges and future directions in this field. To keep track of this field and continuously update our survey, we maintain a repository of relevant references at https://github.com/Paitesanshi/LLM-Agent-Survey.

研究动机与目标

• 解释为何 LLM 能够使自治代理成为现实以及先前工作中的不足。
• 提出一个统一的架构框架，能够涵盖大多数现有的基于 LLM 的代理设计。
• 研究记忆、计划和画像模块，以理解代理的能力与限制。
• 总结在社会科学、自然科学与工程领域的应用，并讨论评估方法。
• 识别挑战与未来方向，以引导基于 LLM 的自治代理的研究。

提出的方法

• 提出一个统一的代理架构，包含画像、记忆、计划和行动模块。
• 将记忆结构分为短期（统一）记忆和具有长期存储与检索的混合记忆。
• 调研带反馈与不带反馈的计划策略，包括单一路径、多路径，以及外部规划器集成。
• 回顾画像策略（手工设计、LLM 生成、数据集对齐）及其对代理行为的影响。
• 总结外部来源与先前工作的以构建分类体系并指导未来发展。

Figure 1: Illustration of the growth trend in the field of LLM-based autonomous agents. We present the cumulative number of papers published from January 2021 to August 2023. We assign different colors to represent various agent categories. For example, a game agent aims to simulate a game-player, w

实验结果

研究问题

• RQ1 哪些架构和模块在利用 LLM 作为自治代理方面最有效？
• RQ2 记忆与计划策略如何提升开放环境下代理的性能与可靠性？
• RQ3 对代理进行画像以使行为与预期角色对齐的最佳实践是什么？
• RQ4 如何评估基于 LLM 的代理？在基准测试中还存在哪些挑战？

主要发现

• 一个统一的四模块框架（画像、记忆、计划、行动）可以涵盖大多数现有基于 LLM 的代理设计。
• 混合记忆，将短期与长期记忆结合，通常用于提升长程推理与一致性。
• 带反馈的计划（环境、人工、模型）相较于无反馈的计划能提高计划的鲁棒性。
• 记忆可以以自然语言、嵌入、数据库或结构化列表的形式存储，且通常会结合多种格式以获得优势。
• 研究者使用单路径与多路径规划、外部规划器，以及基于反馈的循环来处理复杂任务。

Figure 2: A unified framework for the architecture design of LLM-based autonomous agent.