[Paper Review] Ease-of-Teaching and Language Structure from Emergent Communication

Fushan Li, Michael Bowling|arXiv (Cornell University)|Jun 6, 2019

Language and cultural evolution30 references17 citations

TL;DR

This paper introduces a novel training regime that enhances emergent communication by making language easier to teach, using periodic resets of listeners to create abrupt environmental pressure. By forcing agents to re-communicate with new listeners, the method induces more structured, compositional, and teachable languages—particularly when resets are simultaneous rather than staggered, with key improvements in language learnability and topographic similarity.

ABSTRACT

Artificial agents have been shown to learn to communicate when needed to complete a cooperative task. Some level of language structure (e.g., compositionality) has been found in the learned communication protocols. This observed structure is often the result of specific environmental pressures during training. By introducing new agents periodically to replace old ones, sequentially and within a population, we explore such a new pressure -- ease of teaching -- and show its impact on the structure of the resulting language.

Motivation & Objective

• To investigate whether ease-of-teaching can serve as a design principle for shaping emergent communication protocols in multi-agent reinforcement learning.
• To examine how environmental pressures—specifically, the need to teach new listeners—can promote compositional and structured language emergence.
• To compare the effects of simultaneous versus staggered listener resets on language learnability and structural properties.
• To evaluate whether increased policy entropy due to abrupt training shifts leads to more teachable and structured communication.

Proposed method

• A referential game setup is used, where a speaker communicates a target object using a fixed-length message to a listener who selects the correct object from five candidates.
• The speaker’s policy is trained via reinforcement learning with an entropy regularization term to encourage exploration and adaptability.
• New listeners are introduced periodically to the training population, either simultaneously or in a staggered fashion, to simulate a dynamic teaching environment.
• The training regime includes population sizes of 1, 2, and 10 listeners, with success rate and entropy dynamics monitored over time.
• Ease-of-teaching is evaluated by measuring how quickly new listeners can learn the language, using a transfer learning protocol.
• Language structure is quantified using topographic similarity, a measure of compositional structure in emergent communication.

Experimental results

Research questions

• RQ1 Does introducing new listeners periodically lead to emergent languages that are easier to teach?
• RQ2 How does the timing of listener introduction (simultaneous vs. staggered) affect language structure and teachability?
• RQ3 To what extent does abrupt environmental change—induced by listener resets—increase policy entropy and promote compositional language?
• RQ4 Can the pressure of teaching new agents lead to more structured and compressible communication protocols?
• RQ5 Is the presence of a large population beneficial for language learnability, or does it dilute the teaching pressure?

Key findings

• Simultaneous resetting of all listeners results in the highest ease-of-teaching and topographic similarity, indicating more structured and learnable languages.
• Languages trained with periodic resets are significantly easier to teach than those without resets, with the improvement most pronounced under simultaneous resets.
• The entropy of the speaker’s policy increases abruptly upon listener reset, especially in small populations, suggesting a mechanism for language reconfiguration toward teachability.
• Staggered resets in larger populations (e.g., N=10) smooth out training dynamics but reduce the pressure for structural language emergence, weakening the effect.
• The absence of abrupt changes in training objectives—such as in large populations with gradual listener turnover—diminishes the emergence of teachable and structured languages.
• Even in the absence of explicit population pressure, larger populations without resets still show a modest increase in language structure, but far less than under reset regimes.