[论文解读] Augmented Language Models: a Survey

Grégoire Mialon, Roberto Dessì+11 more|arXiv (Cornell University)|Feb 15, 2023

Topic Modeling被引用 140

一句话总结

本综述整理了语言模型如何通过推理和外部工具（ALMs）进行增强，分类方法，并讨论提示、检索和学习方法，以及未来方向和局限性。

ABSTRACT

This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. The former is defined as decomposing a potentially complex task into simpler subtasks while the latter consists in calling external modules such as a code interpreter. LMs can leverage these augmentations separately or in combination via heuristics, or learn to do so from demonstrations. While adhering to a standard missing tokens prediction objective, such augmented LMs can use various, possibly non-parametric external modules to expand their context processing ability, thus departing from the pure language modeling paradigm. We therefore refer to them as Augmented Language Models (ALMs). The missing token objective allows ALMs to learn to reason, use tools, and even act, while still performing standard natural language tasks and even outperforming most regular LMs on several benchmarks. In this work, after reviewing current advance in ALMs, we conclude that this new research direction has the potential to address common limitations of traditional LMs such as interpretability, consistency, and scalability issues.

研究动机与目标

• 定义 Augmented Language Models (ALMs) 并区分推理与工具使用。
• 给出沿三个维度的 ALM 方法学分类：推理、工具与学习策略。
• 回顾提示方法、递归与显式推理，以及工具的整合方式（检索、计算、执行）。
• 调查 ALMs 的学习范式（监督、强化学习、指令微调）并讨论局限性与未来方向。

提出的方法

• 沿三个维度对 ALMs 进行分类：推理、工具使用和学习。
• 描述用于引出推理的提示技术（链式思考、零-shot/少量-shot、Self-Ask、ReAct）。
• 概述递归与显式推理方法（从少到多、将问题分解为子问题）。
• 讨论工具模态，包括调用其他模型、信息检索、符号/代码解释器，以及在虚拟/物理世界中的执行。
• 回顾检索增强语言模型（REALM、RAG、RETRO、Atlas）及其在 LM 上下文中的整合。
• 涵盖学习方法（监督、指令微调、用于推理的微调）及其对推理与可靠性的影响。

实验结果

研究问题

• RQ1 哪些策略能有效引出并增强语言模型的推理？
• RQ2 如何将外部工具与模块整合以扩展 LM 能力（检索、计算、执行）？
• RQ3 哪些学习范式最能提升 ALMs 的推理、工具使用和对齐？
• RQ4 在可解释性、可扩展性和保真性方面，ALMs 的主要局限性和开放方向是什么？

主要发现

• The survey highlights that ALMs can outperform many standard LMs on diverse benchmarks by integrating reasoning, tool use, or both.
• Prominent prompting methods (chain-of-thought, Self-Ask, ReAct) and recursive or explicit reasoning strategies are surveyed for improving multi-step tasks.
• Retrieval-augmented models (REALM, RAG, RETRO, Atlas) are central to expanding LM capabilities with external data sources without excessively enlarging model size.
• There are multiple modalities for tools, including information retrieval, code interpreters, symbolic modules, and inter-model communication, enabling LMs to access up-to-date information and perform computations.
• Learning approaches such as instruction tuning and fine-tuning with reasoning data can make smaller models exhibit reasoning abilities competitive with larger models.
• Despite progress, faithfulness, interpretability of intermediate steps, and efficiency remain key limitations and areas for future work.