[论文解读] From System 1 to System 2: A Survey of Reasoning Large Language Models

Zhongzhi Li, Duzhen Zhang+19 more|ArXiv.org|Feb 24, 2025

Natural Language Processing Techniques被引用 4

一句话总结

本综述回顾从基础 LLM（System 1）到推理型 LLM（System 2）的进展， detailing 核心方法、基准测试与面向更高推理能力的未来方向。它还通过一个实时 GitHub 仓库跟踪发展。

ABSTRACT

Achieving human-level intelligence requires refining the transition from the fast, intuitive System 1 to the slower, more deliberate System 2 reasoning. While System 1 excels in quick, heuristic decisions, System 2 relies on logical reasoning for more accurate judgments and reduced biases. Foundational Large Language Models (LLMs) excel at fast decision-making but lack the depth for complex reasoning, as they have not yet fully embraced the step-by-step analysis characteristic of true System 2 thinking. Recently, reasoning LLMs like OpenAI's o1/o3 and DeepSeek's R1 have demonstrated expert-level performance in fields such as mathematics and coding, closely mimicking the deliberate reasoning of System 2 and showcasing human-like cognitive abilities. This survey begins with a brief overview of the progress in foundational LLMs and the early development of System 2 technologies, exploring how their combination has paved the way for reasoning LLMs. Next, we discuss how to construct reasoning LLMs, analyzing their features, the core methods enabling advanced reasoning, and the evolution of various reasoning LLMs. Additionally, we provide an overview of reasoning benchmarks, offering an in-depth comparison of the performance of representative reasoning LLMs. Finally, we explore promising directions for advancing reasoning LLMs and maintain a real-time \href{https://github.com/zzli2022/Awesome-Slow-Reason-System}{GitHub Repository} to track the latest developments. We hope this survey will serve as a valuable resource to inspire innovation and drive progress in this rapidly evolving field.

研究动机与目标

• Summarize the progression from foundational LLMs to reasoning LLMs and their motivation to achieve System 2-like reasoning.

提出的方法

• Outline foundational LLMs and early System 2 technologies (symbolic logic, MCTS, RL) and their role in reasoning LLMs.
• Describe reasoning LLM construction focusing on output behavior and training dynamics.
• Detail core methods enabling reasoning (Structure Search, Reward Modeling, Self Improvement, Macro Action, Reinforcement Fine-Tuning) and representative models.
• Provide benchmarking coverage and comparative evaluation of reasoning LLMs versus foundational models.
• Highlight limitations and future directions, plus a real-time GitHub tracking resource.

实验结果

研究问题

• RQ1 What are the foundational technologies and their contributions to System 2 reasoning in LLMs?
• RQ2 How can reasoning LLMs be constructed and trained to emulate deliberate, step-by-step reasoning?
• RQ3 What are the main methods and representative models driving modern reasoning capabilities, and how do they compare on benchmarks?
• RQ4 What are the current limitations and potential future directions for reasoning LLMs?

主要发现

• Foundational LLMs enable broad language understanding and emergent capabilities like In-Context Learning and Chain-of-Thought, but remain System 1 in nature.
• Symbolic logic, MCTS, and RL laid the groundwork for reasoning LLMs by providing structured, deliberative frameworks.
• Reasoning LLMs employ core methods such as Structure Search, Reward Modeling, Self Improvement, Macro Action, and Reinforcement Fine-Tuning to achieve advanced reasoning.
• Reasoning benchmarks exist for plain text and multimodal tasks, with comparative evaluations against foundational models.
• The paper discusses limitations and future directions, and maintains a live GitHub repository to track developments.