[Paper Review] LLM-Augmented Agent-Based Modelling for Social Simulations: Challenges and Opportunities

Önder Gürcan|arXiv (Cornell University)|May 8, 2024

Multi-Agent Systems and Negotiation4 citations

TL;DR

This paper proposes a systematic integration of large language models (LLMs) into agent-based modeling (ABM) for social simulations, enabling more realistic, explainable, and interactive simulations of complex social systems. By leveraging LLMs for agent reasoning, dialogue generation, and simulation pipeline support—while addressing challenges like hallucination and explainability—it demonstrates that LLM-augmented ABM enhances model realism, accessibility, and interdisciplinary collaboration.

ABSTRACT

As large language models (LLMs) continue to make significant strides, their better integration into agent-based simulations offers a transformational potential for understanding complex social systems. However, such integration is not trivial and poses numerous challenges. Based on this observation, in this paper, we explore architectures and methods to systematically develop LLM-augmented social simulations and discuss potential research directions in this field. We conclude that integrating LLMs with agent-based simulations offers a powerful toolset for researchers and scientists, allowing for more nuanced, realistic, and comprehensive models of complex systems and human behaviours.

Motivation & Objective

• To address the lack of a conceptual baseline for systematically integrating LLMs into agent-based simulations of social systems.
• To identify and structure key research directions for LLM-augmented ABM that support methodological rigor and scalability.
• To enhance the realism and interpretability of social simulations by leveraging LLMs for agent behavior, decision-making, and natural language explanations.
• To explore the role of LLMs in streamlining simulation workflows, including literature review, data preparation, calibration, and result interpretation.
• To critically assess epistemic risks such as illusion-of-understanding when relying on LLMs as scientific collaborators in simulation research.

Proposed method

• Proposes a conceptual framework for integrating LLMs across the entire ABM pipeline, from agent design to result interpretation.
• Utilizes LLMs as non-deterministic simulators capable of role-playing social agents with beliefs, intentions, and culturally informed behaviors through fine-tuning.
• Employs prompt engineering and conditioning techniques to align LLM outputs with specific social roles, norms, and decision-making patterns.
• Integrates LLM APIs into ABM platforms to enable real-time, interactive dialogue between users and simulated agents.
• Applies LLMs for natural language explanation of agent actions and simulation dynamics, improving transparency and accessibility.
• Emphasizes the need for ethical data handling and tooling platforms grounded in organizational and social system design principles.

Experimental results

Research questions

• RQ1 How can LLMs be systematically integrated into agent-based modeling to enhance the realism and complexity of social simulations?
• RQ2 What are the key methodological challenges in using LLMs for agent reasoning, behavior generation, and simulation workflow support in social systems?
• RQ3 In what ways can LLM-augmented agents improve explainability and accessibility of ABM for non-computational researchers across disciplines?
• RQ4 How do epistemic risks—such as illusion-of-understanding—impact scientific validity when LLMs are used as collaborators in simulation research?
• RQ5 What conceptual and technical architectures are required to ensure reliable, reproducible, and ethically sound LLM-augmented social simulations?

Key findings

• LLM-augmented ABM enables the simulation of socially nuanced agents that can reason, act, and explain their behavior using natural language, enhancing model realism.
• LLMs can streamline simulation workflows by automating literature review, data interpretation, parameter calibration, and sensitivity analysis.
• The integration of LLMs allows for the generation of synthetic populations of social agents that reflect diverse human perspectives and cultural contexts.
• LLMs provide natural language explanations of agent decisions, improving transparency and interpretability for interdisciplinary users.
• Despite benefits, LLM-augmented ABM carries epistemic risks, including the illusion of understanding, where researchers may overestimate the accuracy or depth of LLM-generated insights.
• A structured, organization-oriented conceptual baseline is essential for developing robust, scalable, and ethically compliant LLM-augmented ABM platforms.