[Paper Review] Emergent Communication for Understanding Human Language Evolution: What's Missing?

Lukas Galke, Yoav Ram+1 more|arXiv (Cornell University)|Apr 22, 2022

Language and cultural evolution4 citations

TL;DR

This paper identifies that emergent communication in neural agents fails to replicate key human language evolution phenomena—ease of learning, generalization, and group size effects—due to the absence of human-like cognitive and communicative constraints. It proposes that integrating memory limitations and speaker-listener role alternation in neural models would restore compressibility pressure and promote linguistically plausible compositionality.

ABSTRACT

Emergent communication protocols among humans and artificial neural network agents do not yet share the same properties and show some critical mismatches in results. We describe three important phenomena with respect to the emergence and benefits of compositionality: ease-of-learning, generalization, and group size effects (i.e., larger groups create more systematic languages). The latter two are not fully replicated with neural agents, which hinders the use of neural emergent communication for language evolution research. We argue that one possible reason for these mismatches is that key cognitive and communicative constraints of humans are not yet integrated. Specifically, in humans, memory constraints and the alternation between the roles of speaker and listener underlie the emergence of linguistic structure, yet these constraints are typically absent in neural simulations. We suggest that introducing such communicative and cognitive constraints would promote more linguistically plausible behaviors with neural agents.

Motivation & Objective

• To identify critical discrepancies between emergent communication in humans and neural agents, particularly in the emergence of compositional structure.
• To investigate why neural agents fail to replicate three key human language evolution phenomena: ease of learning, generalization, and group size effects.
• To argue that the absence of human-specific cognitive and communicative constraints—particularly memory limitations and role alternation—undermines the linguistic plausibility of neural emergent communication.
• To propose that integrating memory constraints and shared speaker-listener parameters in neural agents would restore compressibility pressure and enable more human-like language evolution.

Proposed method

• Limit model capacity relative to the message space (|A|^L) to create a compressibility pressure that prevents memorization of idiosyncratic signals.
• Implement shared parameters between speaker and listener roles within a single neural agent to simulate human-like role alternation.
• Use weight tying between input embeddings and output layers to enforce symmetry between production and comprehension in the same network.
• Introduce parameter sharing to encourage the emergence of reusable substructures, promoting compositional generalization.
• Model population dynamics with increasing group sizes to test whether compositional structure emerges under constrained capacity and role alternation.
• Draw on principles from language modeling (e.g., tied embeddings) and iterated learning to formalize the proposed constraints.

Experimental results

Research questions

• RQ1 Why do neural agents fail to replicate the group size effect in emergent communication, despite its robust presence in human communication?
• RQ2 How does the absence of memory constraints in overparameterized neural networks affect the emergence of compositional structure?
• RQ3 To what extent does speaker-listener role alternation influence the learnability and generalization of emergent languages in neural agents?
• RQ4 Can parameter sharing between speaker and listener roles in a single agent simulate the cognitive symmetry observed in human language processing?
• RQ5 What constraints on model capacity are necessary to promote compositionality over memorization in emergent communication?

Key findings

• Neural agents replicate the ease-of-learning effect for compositional languages, but fail to replicate the benefits of generalization and group size effects.
• The group size effect—where larger populations develop more systematic languages—is not consistently reproduced in discrete communication channels with neural agents.
• Recent work shows that population heterogeneity can partially recover the group size effect, but only under specific architectural conditions.
• Overparameterization in neural networks allows them to memorize agent-specific signals, effectively eliminating compressibility pressure and undermining the need for compositionality.
• Memory constraints are shown to be essential for inducing structure in iterated learning and communication games, as they prevent memorization of idiosyncratic signals.
• Role alternation through shared parameters between speaker and listener roles in a single agent promotes more linguistically plausible communication dynamics, mirroring human brain mechanisms.