[Paper Review] STRIDE: A Tool-Assisted LLM Agent Framework for Strategic and Interactive Decision-Making

Chuanhao Li, Runhan Yang+5 more|arXiv (Cornell University)|May 25, 2024

Multi-Agent Systems and Negotiation4 citations

TL;DR

This paper introduces STRIDE, a tool-augmented LLM agent framework that enhances strategic and interactive decision-making in complex environments like bargaining and game theory through structured reasoning, memory, and specialized tools. By integrating algorithms such as Minimax with alpha-beta pruning and using LLMs to orchestrate multi-step planning, STRIDE significantly outperforms baseline LLMs in games like Tic-Tac-Toe and Connect-N, achieving 90% win rate against zero-shot chain-of-thought models.

ABSTRACT

Large Language Models (LLMs) like GPT-4 have revolutionized natural language processing, showing remarkable linguistic proficiency and reasoning capabilities. However, their application in strategic multi-agent decision-making environments is hampered by significant limitations including poor mathematical reasoning, difficulty in following instructions, and a tendency to generate incorrect information. These deficiencies hinder their performance in strategic and interactive tasks that demand adherence to nuanced game rules, long-term planning, exploration in unknown environments, and anticipation of opponents' moves. To overcome these obstacles, this paper presents a novel LLM agent framework equipped with memory and specialized tools to enhance their strategic decision-making capabilities. We deploy the tools in a number of economically important environments, in particular bilateral bargaining and multi-agent and dynamic mechanism design. We employ quantitative metrics to assess the framework's performance in various strategic decision-making problems. Our findings establish that our enhanced framework significantly improves the strategic decision-making capability of LLMs. While we highlight the inherent limitations of current LLM models, we demonstrate the improvements through targeted enhancements, suggesting a promising direction for future developments in LLM applications for interactive environments.

Motivation & Objective

• To address the limitations of LLMs in strategic, interactive, and long-horizon decision-making tasks such as game theory and mechanism design.
• To overcome LLM shortcomings in mathematical reasoning, instruction following, and hallucination by integrating external tools and memory.
• To develop a framework that enables LLMs to perform structured, multi-step reasoning with backward induction and algorithmic simulation.
• To evaluate the framework in economically relevant domains including bilateral bargaining, dynamic mechanism design, and sequential games.
• To demonstrate that tool-augmented LLM agents can achieve near-optimal performance in strategic environments where standard prompting fails.

Proposed method

• STRIDE uses a controller LLM to orchestrate a structured Thought sequence involving planning, tool use, and reflection, enabling multi-step reasoning.
• The framework integrates operational tools such as CalculateScores and GetScores to simulate algorithmic processes like Breadth-First Minimax with alpha-beta pruning.
• It employs backward induction and working memory to store and retrieve computed scores for game states, enabling optimal move selection.
• The system is designed to emulate reference algorithms such as value iteration, upper confidence bound value iteration, and backward induction in different strategic settings.
• It supports both known and unknown environment models by dynamically switching between planning with known dynamics and exploration with tool-assisted simulation.
• The framework is evaluated using prompts tailored to specific tasks, including Markov decision processes, dynamic mechanism design, and bargaining games.

Experimental results

Research questions

• RQ1 Can an LLM agent with tool augmentation and memory outperform standard prompting in strategic decision-making tasks?
• RQ2 How effective is the STRIDE framework in environments requiring long-horizon planning and opponent modeling, such as sequential bargaining?
• RQ3 To what extent can LLMs simulate complex algorithms like Minimax with alpha-beta pruning through tool-assisted reasoning?
• RQ4 Does the integration of memory and structured tool use improve performance in games like Tic-Tac-Toe and Connect-N compared to zero-shot chain-of-thought prompting?
• RQ5 Can the framework generalize across different strategic domains, including dynamic mechanism design and bilateral bargaining?

Key findings

• In Tic-Tac-Toe, STRIDE achieved a 90% win rate against zero-shot chain-of-thought models and an 80% win rate against zero-shot chain-of-thought with code.
• STRIDE won 50% of games against RAFA with Monte Carlo Tree Search (MCTS), indicating strong performance even against specialized agents.
• In Connect-3, STRIDE achieved a 30% win rate, significantly outperforming zero-shot CoT (60% wins) and zero-shot CoT with code (90% wins), suggesting strategic consistency.
• In Connect-4, STRIDE achieved a 50% win rate, matching the baseline zero-shot CoT with code and outperforming zero-shot CoT in tie outcomes (50% vs. 10% ties).
• STRIDE achieved 80% tie rate in Tic-Tac-Toe across 10 runs, indicating strong stability and near-optimality in symmetric games.
• The framework demonstrated consistent performance across multiple strategic domains, including Markov decision processes, dynamic mechanism design, and bargaining games, with measurable improvements over baseline LLMs.