[Paper Review] Human vs. Machine: Behavioral Differences Between Expert Humans and Language Models in Wargame Simulations

Max Lamparth, Anthony Corso+4 more|arXiv (Cornell University)|Mar 6, 2024

Multi-Agent Systems and Negotiation4 citations

TL;DR

This study compares expert human decision-making with large language model (LLM)-simulated responses in a U.S.-China wargame simulating crisis escalation in the Taiwan Strait. Using a 21-action decision framework, the authors find significant overlap between human and LLM responses—especially when LLMs are prompted to simulate dialogue—but also systematic differences in aggression, action selection, and sensitivity to input instructions, highlighting risks in relying on LLMs for high-stakes strategic recommendations without human oversight.

ABSTRACT

To some, the advent of artificial intelligence (AI) promises better decision-making and increased military effectiveness while reducing the influence of human error and emotions. However, there is still debate about how AI systems, especially large language models (LLMs) that can be applied to many tasks, behave compared to humans in high-stakes military decision-making scenarios with the potential for increased risks towards escalation. To test this potential and scrutinize the use of LLMs for such purposes, we use a new wargame experiment with 214 national security experts designed to examine crisis escalation in a fictional U.S.-China scenario and compare the behavior of human player teams to LLM-simulated team responses in separate simulations. Here, we find that the LLM-simulated responses can be more aggressive and significantly affected by changes in the scenario. We show a considerable high-level agreement in the LLM and human responses and significant quantitative and qualitative differences in individual actions and strategic tendencies. These differences depend on intrinsic biases in LLMs regarding the appropriate level of violence following strategic instructions, the choice of LLM, and whether the LLMs are tasked to decide for a team of players directly or first to simulate dialog between a team of players. When simulating the dialog, the discussions lack quality and maintain a farcical harmony. The LLM simulations cannot account for human player characteristics, showing no significant difference even for extreme traits, such as "pacifist" or "aggressive sociopath." When probing behavioral consistency across individual moves of the simulation, the tested LLMs deviated from each other but generally showed somewhat consistent behavior. Our results motivate policymakers to be cautious before granting autonomy or following AI-based strategy recommendations.

Motivation & Objective

• To evaluate how closely LLM-simulated responses mirror those of expert human players in a high-stakes U.S.-China crisis wargame.
• To investigate how variations in LLM prompting (e.g., dialogue simulation vs. direct action selection) affect strategic behavior and outcome predictability.
• To assess the extent to which LLMs can accurately replicate human psychological and strategic preferences, including background attributes and personal biases.
• To identify systematic deviations between LLM and human behavior in crisis escalation scenarios, particularly regarding aggression and rules of engagement.
• To inform policymakers about the risks of deploying LLMs in autonomous military decision-making without rigorous validation.

Proposed method

• Conducted a 2-move wargame with 107 national security experts simulating U.S. National Security Council decisions in a fictional 2026 U.S.-China crisis near the Taiwan Strait.
• Used LLMs (including GPT-4) to simulate responses under two prompting conditions: (1) role-playing dialogue between players, and (2) direct instruction to list actions for each player role.
• Collected and compared response vectors across 21 possible actions, using linear discriminant analysis to visualize distributional similarity between human and LLM responses.
• Analyzed qualitative differences in reasoning, escalation tendencies, and strategic framing between human and LLM-generated responses.
• Evaluated the impact of LLM input instructions on behavioral outcomes, particularly in terms of aggression and action count.
• Assessed the LLMs’ inability to account for player-specific attributes such as background, personal preferences, or institutional roles.

Experimental results

Research questions

• RQ1 How do LLM-simulated responses compare quantitatively and qualitatively to expert human responses in a U.S.-China crisis wargame?
• RQ2 To what extent does the prompting format (dialogue simulation vs. direct action selection) influence LLM behavior and strategic outcomes?
• RQ3 Can LLMs accurately replicate human psychological and strategic preferences, including background-specific decision-making?
• RQ4 What are the systematic differences in escalation tendencies between human and LLM players, particularly in rules of engagement?
• RQ5 How do variations in LLM architecture and instruction design affect the reliability and safety of AI-generated strategic recommendations?

Key findings

• LLM-simulated responses showed significant overlap with human responses, with agreement on approximately half of the 21 possible actions in the wargame.
• When prompted to simulate dialogue between players, LLM responses lacked meaningful interaction and failed to reflect dynamic negotiation or strategic adaptation.
• LLM responses became more aggressive and selected more actions when instructed to directly state actions, compared to dialogue-simulation prompts.
• The LLM demonstrated no ability to account for player background attributes, personal preferences, or institutional roles, leading to generic and context-insensitive recommendations.
• Different LLMs produced varying levels of aggression and decision-making patterns, indicating sensitivity to model choice and prompting design.
• Despite surface-level similarity in action selection, qualitative differences in reasoning and escalation framing suggest LLMs may misrepresent human strategic intent in high-stakes scenarios.