[Paper Review] Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input

Angeliki Lazaridou, Karl Moritz Hermann+2 more|arXiv (Cornell University)|Apr 11, 2018

Language and cultural evolution40 citations

TL;DR

The paper trains reinforcement-learning agents to communicate in referential games, first with symbolic, then with pixel inputs, showing that structured input fosters more compositional language and that raw-pixel inputs pose challenges for grounding and structure emergence.

ABSTRACT

The ability of algorithms to evolve or learn (compositional) communication protocols has traditionally been studied in the language evolution literature through the use of emergent communication tasks. Here we scale up this research by using contemporary deep learning methods and by training reinforcement-learning neural network agents on referential communication games. We extend previous work, in which agents were trained in symbolic environments, by developing agents which are able to learn from raw pixel data, a more challenging and realistic input representation. We find that the degree of structure found in the input data affects the nature of the emerged protocols, and thereby corroborate the hypothesis that structured compositional language is most likely to emerge when agents perceive the world as being structured.

Motivation & Objective

• Investigate how environmental structure and input representation affect emergent communication in referential games.
• Extend prior symbolic-input studies to raw pixel inputs to assess realism and grounding in language emergence.
• Examine whether and how compositional structure emerges under different input regimes and task settings.

Proposed method

• Model two cooperative agents (speaker and listener) trained with reinforcement learning to maximize successful identification of a target object in a referential game.
• Represent objects either as binary attribute vectors (symbolic, disentangled) or as raw pixel images (entangled).
• Speaker encodes target into a dense vector u, then outputs a discrete, variable-length message m from an alphabet A via an LSTM decoder.
• Listener encodes candidate objects into dense representations and uses a separate LSTM to process the message into z, then selects the target via a Gibbs-based pointing module.
• Jointly optimize all weights with REINFORCE, including an entropy regularizer to maintain exploration.
• Compare performance, generalization to novel objects, and topographic similarity between meaning and signal spaces.

Experimental results

Research questions

• RQ1 Does a communicative protocol emerge when agents are trained end-to-end on referential games with symbolic vs. pixel inputs?
• RQ2 How does input structure (disentangled attributes vs. entangled pixel data) influence the emergence of compositionality and generalization?
• RQ3 What is the effect of distractor distribution and contextual co-occurrence on the learned language and its structure?
• RQ4 Can emergent protocols from pixel inputs exhibit topographic similarity between meanings and signals?

Key findings

• Agents achieve high communicative success in both symbolic and pixel-input settings (up to around 98% on training data under longer messages).
• Longer maximum message lengths induce larger lexicons and more one-to-one mappings between messages and concepts, reducing ambiguity.
• In symbolic input, emergent messages show positive topographic similarity, with prefixes encoding category information (e.g., mammal, vehicle).
• With pixel input, agents also attain high accuracy (e.g., 93.7% in game A) and exhibit topographic similarity, but protocols tend to be unstable and highly dependent on game constraints, sometimes producing ad-hoc naming conventions.
• Disentanglement is identified as a key factor enabling more structured language; when input is entangled (pixel data), organizing factors of variation is harder, reducing compositionality.
• Contextual distractor distributions influence language learning dynamics and the degree of generalization, with context-dependent distractors sometimes delaying convergence but potentially reducing confusability driven by similarity.
• In pixel-based setups, probing shows that visual representations encode location information consistently, with varying predictive power for color and shape across games.