[Paper Review] GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations

Jinhao Duan, Renming Zhang+7 more|arXiv (Cornell University)|Feb 19, 2024

Auction Theory and ApplicationsDecision Sciences3 citations

TL;DR

GTBench introduces a comprehensive game-theoretic benchmark evaluating Large Language Models (LLMs) across 10 diverse tasks spanning complete/incomplete information, deterministic/probabilistic, and static/dynamic games. It reveals that LLMs fail in complete and deterministic games like Nim and Tic-Tac-Toe but remain competitive in probabilistic or incomplete-information settings such as Kuhn Poker, with code-pretraining boosting performance while advanced reasoning techniques like Chain-of-Thought do not always help.

ABSTRACT

As Large Language Models (LLMs) are integrated into critical real-world applications, their strategic and logical reasoning abilities are increasingly crucial. This paper evaluates LLMs' reasoning abilities in competitive environments through game-theoretic tasks, e.g., board and card games that require pure logic and strategic reasoning to compete with opponents. We first propose GTBench, a language-driven environment composing 10 widely recognized tasks, across a comprehensive game taxonomy: complete versus incomplete information, dynamic versus static, and probabilistic versus deterministic scenarios. Then, we (1) Characterize the game-theoretic reasoning of LLMs; and (2) Perform LLM-vs.-LLM competitions as reasoning evaluation. We observe that (1) LLMs have distinct behaviors regarding various gaming scenarios; for example, LLMs fail in complete and deterministic games yet they are competitive in probabilistic gaming scenarios; (2) Most open-source LLMs, e.g., CodeLlama-34b-Instruct and Llama-2-70b-chat, are less competitive than commercial LLMs, e.g., GPT-4, in complex games, yet the recently released Llama-3-70b-Instruct makes up for this shortcoming. In addition, code-pretraining greatly benefits strategic reasoning, while advanced reasoning methods such as Chain-of-Thought (CoT) and Tree-of-Thought (ToT) do not always help. We further characterize the game-theoretic properties of LLMs, such as equilibrium and Pareto Efficiency in repeated games. Detailed error profiles are provided for a better understanding of LLMs' behavior. We hope our research provides standardized protocols and serves as a foundation to spur further explorations in the strategic reasoning of LLMs.

Motivation & Objective

• To address the lack of rigorous, logic-focused evaluation environments for LLM reasoning beyond narrative-based role-playing games.
• To characterize how LLMs perform across diverse game-theoretic scenarios, especially in terms of strategic and logical reasoning.
• To establish LLM-vs-LLM competitions as a new benchmark for evaluating reasoning capabilities in competitive, interactive settings.
• To identify key factors influencing LLM performance in strategic games, such as model type, pretraining, and reasoning techniques.

Proposed method

• Designing GTBench, a language-driven environment with 10 widely recognized game-theoretic tasks across a comprehensive taxonomy of game types.
• Classifying games along dimensions: complete vs. incomplete information, deterministic vs. probabilistic, static vs. dynamic.
• Conducting two evaluation configurations: LLM-vs-Conventional (e.g., MCTS solvers) and LLM-vs-LLM competitions to assess reasoning under competition.
• Using regret value metrics to quantitatively assess decision quality in games like Blind Auction and Iterated Prisoner’s Dilemma.
• Analyzing error patterns via detailed error profiles to diagnose failures in interpretation, calculation, and overconfidence.
• Employing code-pretraining and reasoning techniques like Chain-of-Thought and Tree-of-Thought to evaluate their impact on strategic performance.

Experimental results

Research questions

• RQ1 How do LLMs perform across different game-theoretic scenarios, particularly in complete vs. incomplete information and deterministic vs. probabilistic games?
• RQ2 To what extent do advanced reasoning methods like Chain-of-Thought and Tree-of-Thought improve LLM performance in strategic games?
• RQ3 How does code-pretraining influence strategic reasoning abilities in LLMs compared to standard LLMs?
• RQ4 How do open-source LLMs compare to commercial LLMs in competitive, rule-based strategic environments?
• RQ5 What are the dominant error patterns in LLM reasoning during game-theoretic tasks, and how do they affect decision quality?

Key findings

• LLMs consistently fail against MCTS solvers in complete and deterministic games such as Nim and Tic-Tac-Toe, indicating a fundamental limitation in pure logical reasoning.
• LLMs remain competitive in incomplete-information and probabilistic games like Kuhn Poker and Negotiation, suggesting better adaptation to uncertainty than to deterministic logic.
• Code-pretrained LLMs like CodeLlama-34b-Instruct achieve performance comparable to GPT-3.5-turbo, demonstrating that code pretraining enhances strategic reasoning.
• Advanced reasoning techniques such as Chain-of-Thought and Tree-of-Thought do not consistently improve performance, and in some cases, degrade it.
• Open-source LLMs underperform commercial models like GPT-4 in complex games with large action/state spaces, but perform comparably in simpler games with limited options.
• Common error patterns include factual errors (e.g., misjudging game state), calculation errors (e.g., incorrect XOR in Nim), misinterpretation of board positions, and overconfidence in opponent cooperation.