[Paper Review] Augmented Language Models: a Survey

Grégoire Mialon, Roberto Dessì+11 more|arXiv (Cornell University)|Feb 15, 2023

Topic ModelingComputer Science140 citations

TL;DR

This survey catalogs how language models are augmented with reasoning and external tools (ALMs), classifies methods, and discusses prompting, retrieval, and learning approaches, along with future directions and limitations.

ABSTRACT

This survey reviews works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. The former is defined as decomposing a potentially complex task into simpler subtasks while the latter consists in calling external modules such as a code interpreter. LMs can leverage these augmentations separately or in combination via heuristics, or learn to do so from demonstrations. While adhering to a standard missing tokens prediction objective, such augmented LMs can use various, possibly non-parametric external modules to expand their context processing ability, thus departing from the pure language modeling paradigm. We therefore refer to them as Augmented Language Models (ALMs). The missing token objective allows ALMs to learn to reason, use tools, and even act, while still performing standard natural language tasks and even outperforming most regular LMs on several benchmarks. In this work, after reviewing current advance in ALMs, we conclude that this new research direction has the potential to address common limitations of traditional LMs such as interpretability, consistency, and scalability issues.

Motivation & Objective

• Define Augmented Language Models (ALMs) and distinguish reasoning and tool usage.
• Provide a taxonomy of ALM approaches along three axes: reasoning, tools, and learning strategies.
• Review prompting methods, recursive and explicit reasoning, and how tools are integrated (retrieval, computation, act).
• Survey learning paradigms (supervision, reinforcement learning, instruction fine-tuning) for ALMs and discuss limitations and future directions.

Proposed method

• Classify ALMs along three axes: reasoning, tool use, and learning.
• Describe prompting techniques to elicit reasoning (chain-of-thought, zero-shot/few-shot, Self-Ask, ReAct).
• Summarize recursive and explicit reasoning methods (least-to-most, decomposition into subproblems).
• Discuss tool modalities including calling other models, information retrieval, symbolic/code interpreters, and actuation in virtual/physical worlds.
• Review retrieval-augmented language models (REALM, RAG, RETRO, Atlas) and their integration into LM context.
• Cover learning approaches (supervision, instruction tuning, fine-tuning for reasoning) and their impact on reasoning and reliability.

Experimental results

Research questions

• RQ1 What strategies effectively elicit and enhance reasoning in language models?
• RQ2 How can external tools and modules be integrated to extend LM capabilities (retrieval, computation, action)?
• RQ3 What learning paradigms best improve reasoning, tool usage, and alignment in ALMs?
• RQ4 What are the main limitations and open directions for ALMs in terms of interpretability, scalability, and fidelity?],
• RQ5 key_findings v1:0
• RQ6 3-6 concise bullets on the main quantitative results.
• RQ7 table_headers:[],
• RQ8 table_rows:[] }%0A</GeneratedReview>
• RQ9 table_headers:[],
• RQ10 table_rows:[] }%0A</GeneratedReview>
• RQ11 table_headers:[],
• RQ12 table_rows:[] }%0A</GeneratedReview>
• RQ13 table_headers:[],
• RQ14 table_rows:[] }%0A</GeneratedReview>
• RQ15 table_headers:[],
• RQ16 table_rows:[] }%0A</GeneratedReview>
• RQ17 table_headers:[],
• RQ18 table_rows:[] }%0A</GeneratedReview>
• RQ19 table_headers:[],
• RQ20 table_rows:[] }%0A</GeneratedReview>
• RQ21 table_headers:[],
• RQ22 table_rows:[] }%0A</GeneratedReview>
• RQ23 table_headers:[],
• RQ24 table_rows:[] }%0A</GeneratedReview>
• RQ25 table_headers:[],
• RQ26 table_rows:[] }%0A</GeneratedReview>
• RQ27 table_headers:[],
• RQ28 table_rows:[] }%0A</GeneratedReview>,

Key findings

• The survey highlights that ALMs can outperform many standard LMs on diverse benchmarks by integrating reasoning, tool use, or both.
• Prominent prompting methods (chain-of-thought, Self-Ask, ReAct) and recursive or explicit reasoning strategies are surveyed for improving multi-step tasks.
• Retrieval-augmented models (REALM, RAG, RETRO, Atlas) are central to expanding LM capabilities with external data sources without excessively enlarging model size.
• There are multiple modalities for tools, including information retrieval, code interpreters, symbolic modules, and inter-model communication, enabling LMs to access up-to-date information and perform computations.
• Learning approaches such as instruction tuning and fine-tuning with reasoning data can make smaller models exhibit reasoning abilities competitive with larger models.
• Despite progress, faithfulness, interpretability of intermediate steps, and efficiency remain key limitations and areas for future work.