[Paper Review] Coordination Among Neural Modules Through a Shared Global Workspace

Anirudh Goyal, Aniket Didolkar+8 more|arXiv (Cornell University)|Mar 1, 2021

Ferroelectric and Negative Capacitance DevicesEngineering51 references17 citations

TL;DR

This paper proposes a shared global workspace with limited bandwidth to coordinate modular neural networks, enabling specialists to compete for attention and broadcast integrated representations. The method improves performance on visual reasoning tasks by enforcing selective, coherent communication, outperforming pairwise attention mechanisms in object tracking, physical reasoning, and Atari game transfer learning.

ABSTRACT

Deep learning has seen a movement away from representing examples with a monolithic hidden state towards a richly structured state. For example, Transformers segment by position, and object-centric architectures decompose images into entities. In all these architectures, interactions between different elements are modeled via pairwise interactions: Transformers make use of self-attention to incorporate information from other positions; object-centric architectures make use of graph neural networks to model interactions among entities. However, pairwise interactions may not achieve global coordination or a coherent, integrated representation that can be used for downstream tasks. In cognitive science, a global workspace architecture has been proposed in which functionally specialized components share information through a common, bandwidth-limited communication channel. We explore the use of such a communication channel in the context of deep learning for modeling the structure of complex environments. The proposed method includes a shared workspace through which communication among different specialist modules takes place but due to limits on the communication bandwidth, specialist modules must compete for access. We show that capacity limitations have a rational basis in that (1) they encourage specialization and compositionality and (2) they facilitate the synchronization of otherwise independent specialists.

Motivation & Objective

• To address the challenge of coordinating functionally specialized neural modules in deep learning architectures.
• To improve coherence and integration of representations across modules without relying on pairwise interactions.
• To explore how limited communication bandwidth can promote specialization and compositionality in modular networks.
• To enable better generalization and transfer learning through a centralized, bottlenecked communication mechanism.
• To bridge classical modular AI architectures with modern deep learning via differentiable, end-to-end training.

Proposed method

• A shared global workspace acts as a bandwidth-limited communication channel where specialist modules compete to write information.
• Specialist modules process inputs independently before selectively writing to the workspace based on relevance.
• The updated workspace contents are broadcast to all specialists, enabling synchronized updates across the network.
• The system uses differentiable attention mechanisms to allow gradient-based training of the entire architecture.
• The workspace's limited capacity enforces selective attention, promoting efficient and coherent information sharing.
• The architecture is applied to models like SCOFF and RIMs, with shared workspace attention replacing or augmenting standard self-attention.

Experimental results

Research questions

• RQ1 How can modular neural networks achieve coherent, integrated representations without relying on pairwise interactions?
• RQ2 What role does limited communication bandwidth play in promoting specialization and coordination among neural specialists?
• RQ3 Can a shared workspace improve performance on visual reasoning and physical prediction tasks compared to standard self-attention?
• RQ4 Does the shared workspace mechanism enhance transfer learning in multi-environment settings like Atari games?
• RQ5 How does the number of slots in the shared workspace affect model performance and generalization?

Key findings

• The shared workspace model achieved an Adjusted Rand Index (ARI) of 0.915 on the bouncing balls task with 5 workspace slots, significantly outperforming SCOFF without a shared workspace (ARI: 0.276).
• With 5 workspace slots, the model reduced mean squared error (MSE) to 0.035, a 58% improvement over SCOFF’s 0.083 MSE.
• Increasing the number of workspace slots beyond 5 led to performance degradation, validating the benefit of bandwidth-limited communication.
• On Atari games, RIMs augmented with the shared workspace achieved a median performance ratio of 1.13 on game A and 1.11 on game B, indicating improved forward transfer and reduced backward interference.
• The model converged faster and outperformed baselines on both relational and non-relational questions in visual reasoning benchmarks.
• The shared workspace enabled better object segregation and coherent physical reasoning by focusing attention on relevant entities through selective broadcasting.