[論文レビュー] LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models

Yadong Zhang, Shaoguang Mao+8 more|arXiv (Cornell University)|Apr 1, 2024

Natural Language Processing Techniques被引用数 6

ひとこと要約

A comprehensive survey of how LLMs are used for strategic reasoning in multi-agent settings, including taxonomy, applications, methods, evaluation, and challenges.

ABSTRACT

This paper presents a comprehensive survey of the current status and opportunities for Large Language Models (LLMs) in strategic reasoning, a sophisticated form of reasoning that necessitates understanding and predicting adversary actions in multi-agent settings while adjusting strategies accordingly. Strategic reasoning is distinguished by its focus on the dynamic and uncertain nature of interactions among multi-agents, where comprehending the environment and anticipating the behavior of others is crucial. We explore the scopes, applications, methodologies, and evaluation metrics related to strategic reasoning with LLMs, highlighting the burgeoning development in this area and the interdisciplinary approaches enhancing their decision-making performance. It aims to systematize and clarify the scattered literature on this subject, providing a systematic review that underscores the importance of strategic reasoning as a critical cognitive capability and offers insights into future research directions and potential improvements.

研究の動機と目的

• Define strategic reasoning with LLMs and distinguish it from other reasoning tasks.
• Survey scenario categories and applications of LLM-based strategic reasoning across societies, economics, games, and gaming.
• Review methods to enhance LLM strategic reasoning (prompting, modular upgrades, theory of mind, imitation/reinforcement learning).
• Discuss evaluation approaches and metrics for strategic reasoning in LLMs and identify current challenges and future directions.

提案手法

• Propose a formal definition and core characteristics of strategic reasoning with LLMs.
• Develop a taxonomy of strategic reasoning scenarios (Societal Simulation, Economic Simulation, Game Theory, Gaming).
• Classify and synthesize methods to improve strategic reasoning (Prompt Engineering, Modular Enhancements, Theory of Mind, Imitation Learning and RL with LLMs).
• Outline evaluation frameworks including outcome-based metrics (wins, rewards) and process-based assessments (prediction accuracy, nonstationarity handling) and discuss qualitative analysis.
• Highlight existing benchmarking efforts (GTBench, LLMArena) and the need for unified benchmarks.

実験結果

リサーチクエスチョン

• RQ1 What constitutes strategic reasoning when performed by LLMs in multi-agent settings?
• RQ2 What scenarios and domains most benefit from LLM-based strategic reasoning (societal, economic, game-theoretic, gaming)?
• RQ3 What methods and architectures best enhance LLM strategic reasoning?
• RQ4 How should we evaluate LLM strategic reasoning, and what benchmarks exist or are needed?
• RQ5 What are the challenges and opportunities for future research in this area?

主な発見

• LLMs enable strategic reasoning across diverse scenarios: societal, economic, game theory, and gaming domains.
• Four broadly categorized methodological families (Prompt Engineering, Modular Enhancements, Theory of Mind, Imitation Learning and RL) are used to boost strategic reasoning and can be combined.
• Evaluation combines outcome-centric metrics (wins, rewards, NRA, TrueSkill) with process-oriented and qualitative assessments (predicting opponents, deception/cooperation, explainability).
• There is evidence that Theory of Mind and opponent-specific adaptations improve predictive accuracy and decision quality in strategic tasks.
• There is a recognized need for unified benchmarks to compare diverse approaches and capture broad strategic reasoning capabilities.