[Paper Review] Compositional Languages Emerge in a Neural Iterated Learning Model

Yi Ren, Shangmin Guo+3 more|arXiv (Cornell University)|Feb 4, 2020

Language and cultural evolution24 references34 citations

TL;DR

The paper introduces a neural iterated learning (NIL) algorithm showing that languages with high topological similarity (rho) emerge in neural agents' referential games, improving generalization and learning speed, with a probabilistic analysis of NIL.

ABSTRACT

The principle of compositionality, which enables natural language to represent complex concepts via a structured combination of simpler ones, allows us to convey an open-ended set of messages using a limited vocabulary. If compositionality is indeed a natural property of language, we may expect it to appear in communication protocols that are created by neural agents in language games. In this paper, we propose an effective neural iterated learning (NIL) algorithm that, when applied to interacting neural agents, facilitates the emergence of a more structured type of language. Indeed, these languages provide learning speed advantages to neural agents during training, which can be incrementally amplified via NIL. We provide a probabilistic model of NIL and an explanation of why the advantage of compositional language exist. Our experiments confirm our analysis, and also demonstrate that the emerged languages largely improve the generalizing power of the neural agent communication.

Motivation & Objective

• Investigate whether high topological similarity (rho) languages emerge in neural agent communication.
• Quantify learning speed advantages of compositional languages for speaking and listening agents.
• Develop NIL (neural iterated learning) to amplify compositional languages across generations.
• Provide a probabilistic framework explaining NIL mechanisms and its robustness.

Proposed method

• Model a referential game with speaking (Alice) and listening (Bob) neural agents using MLPs and LSTMs.
• Define messages as fixed-length discrete sequences from a vocabulary; objects encoded as one-hot attribute vectors.
• Train with REINFORCE and entropy regularization; provide gradient expressions for Alice and Bob (equations (1) and (2)).
• Measure compositionality via topological similarity rho between object space distances and message space distances (negative cosine in X, Levenshtein in M).
• Propose a three-phase NIL algorithm (Learning, Interacting, Transmitting) over I generations to selectively reinforce high-rho languages.

Experimental results

Research questions

• RQ1 Do high-rho languages confer learning speed advantages to neural speakers and listeners?
• RQ2 Can NIL amplify the emergence of compositional languages and improve generalization in neural agents?
• RQ3 What is the role of pre-training and generation resets in driving rho toward higher values?
• RQ4 How does topological similarity correlate with validation performance on held-out items?

Key findings

• High rho languages provide learning speed advantages for both Alice and Bob in a referential game.
• NIL increases the topological similarity between emergent language and meaning space across generations.
• Iterated learning procedures yield higher rho and better validation performance than non-reset training, with robustness across settings.
• There is a strong positive correlation between rho and validation performance (correlation = 0.928, p = 3.8e-104).
• An optimal interval for pre-training rounds exists (I_a and I_b) that maximizes rho growth and task performance.