[Paper Review] AI agents can coordinate beyond human scale

Giordano De Marzo, Claudio Castellano+1 more|arXiv (Cornell University)|Sep 4, 2024

Translation Studies and PracticesArts and Humanities3 citations

TL;DR

This paper investigates whether large language model (LLM) societies can achieve consensus on arbitrary norms without external guidance, using a framework inspired by statistical physics and behavioral science. It finds that consensus emerges via a 'majority force' proportional to LLMs' language understanding, enabling groups to reach consensus at scales far exceeding human limits—up to orders of magnitude larger than typical human groups—when language capabilities are high enough.

ABSTRACT

Large language models (LLMs) are increasingly deployed in collaborative tasks involving multiple agents, forming an "AI agent society: where agents interact and influence one another. Whether such groups can spontaneously coordinate on arbitrary decisions without external influence - a hallmark of self-organized regulation in human societies - remains an open question. Here we investigate the stability of groups formed by AI agents by applying methods from complexity science and principles from behavioral sciences. We find that LLMs can spontaneously form cohesive groups, and that their opinion dynamics is governed by a majority force coefficient, which determines whether coordination is achievable. This majority force diminishes as group size increases, leading to a critical group size beyond which coordination becomes practically unattainable and stability is lost. Notably, this critical group size grows exponentially with the language capabilities of the models, and for the most advanced LLMs, it exceeds the typical size of informal human groups. Our findings highlight intrinsic limitations in the self-organization of AI agent societies and have implications for the design of collaborative AI systems where coordination is desired or could represent a treat.

Motivation & Objective

• To investigate whether LLMs can achieve consensus on arbitrary norms without external guidance or information favoring one option.
• To understand the limits of collective coordination in LLM societies, analogous to cognitive limits in human groups.
• To quantify how language understanding capabilities of LLMs affect their ability to reach consensus in large groups.
• To model opinion dynamics in LLM ensembles using principles from statistical physics and behavioral science.
• To determine the critical group size beyond which consensus becomes unattainable for a given LLM, based on its language understanding capacity.

Proposed method

• Simulate opinion dynamics in groups of LLM agents using a binary opinion update protocol where each agent selects an opinion based on the collective opinion of others.
• Use a prompt-based interaction framework where agents are shown a list of peers and their opinions, then respond with a choice between two arbitrary opinion labels (e.g., 'k' and 'z') to avoid intrinsic biases.
• Apply a random shuffling of opinion labels at each time step to eliminate systematic preference for one label, ensuring fairness in evaluation.
• Model the system using a Curie-Weiss (CW) mean-field spin model, mapping collective opinion to magnetization 'm' and using the majority force 'β' as a key parameter.
• Fit empirical opinion transition probabilities to the function P(m) = 0.5[tanh(β·m) + 1], confirming universal behavior across models.
• Derive the self-consistency equation m = tanh(β·m) to determine the critical point β = 1, marking the onset of consensus formation.

Experimental results

Research questions

• RQ1 Can LLMs reach consensus on arbitrary norms in the absence of external information or preference?
• RQ2 What is the maximum group size for which consensus is feasible, given a specific LLM's language understanding capability?
• RQ3 How does the 'majority force' β—driven by language understanding—affect the emergence of consensus in LLM groups?
• RQ4 Is there a critical group size beyond which consensus becomes statistically unattainable for a given LLM?
• RQ5 Do the opinion dynamics of LLMs follow a universal functional form, independent of model architecture or identity?

Key findings

• The majority force β, which governs consensus formation, increases with the language understanding capability of LLMs, enabling larger-scale coordination.
• Consensus becomes unfeasible beyond a critical group size N_c, where β(N_c) = 1, marking a second-order phase transition analogous to the Curie-Weiss model.
• For the most advanced LLMs, the critical group size N_c grows exponentially with language understanding, reaching orders of magnitude beyond typical human group sizes (e.g., ~250 individuals).
• Empirical opinion dynamics across diverse LLMs (GPT, Claude, Llama) collapse onto a universal function P(m) = 0.5[tanh(β·m) + 1], confirming a shared underlying dynamic.
• The system exhibits a phase transition at β = 1, where consensus emerges gradually as β increases beyond this threshold.
• Statistical fluctuations prevent full consensus even when β > 1, making β(N_c) = 1 a strict upper bound on feasible consensus group size.