[论文解读] Recurrent Independent Mechanisms

Anirudh Goyal, Alex Lamb+5 more|arXiv (Cornell University)|Sep 24, 2019

Modular Robots and Swarm Intelligence参考文献 62被引用 80

一句话总结

我们引入 Recurrent Independent Mechanisms (RIMs)，一种模块化循环结构，其中多个独立机制通过注意力以稀疏方式交互，以在分布变化下改善泛化。

ABSTRACT

Learning modular structures which reflect the dynamics of the environment can lead to better generalization and robustness to changes which only affect a few of the underlying causes. We propose Recurrent Independent Mechanisms (RIMs), a new recurrent architecture in which multiple groups of recurrent cells operate with nearly independent transition dynamics, communicate only sparingly through the bottleneck of attention, and are only updated at time steps where they are most relevant. We show that this leads to specialization amongst the RIMs, which in turn allows for dramatically improved generalization on tasks where some factors of variation differ systematically between training and evaluation.

研究动机与目标

• Motivate learning modular structures that reflect environment dynamics to improve generalization and robustness to changes affecting only some causes.
• Propose RIMs, a recurrent architecture with near-independent mechanisms that communicate sparsely and update only when relevant.
• Show that specialization among RIMs yields better generalization on tasks with varying factors of variation between training and evaluation.

提出的方法

• Divide the model into k recurrent mechanisms (RIMs) with shared parameters per mechanism.
• Activate and update only a subset of RIMs per time step via input-driven top-k attention over RIMs.
• Each activated RIM applies its own independent transition dynamics (e.g., GRU/LSTM) and can read from other RIMs via a residual, sparse communication attention.
• Use key-value attention to read from input and to enable sparse inter-RIM communication through queries from RIM states and keys/values from inputs and other RIMs.
• Optionally apply spatial sparsity by computing attention per spatial position when inputs are structured (e.g., images).
• Provide multi-head attention for input and communication, with per-RIM learned parameters governing attention.

实验结果

研究问题

• RQ1 Can a gradient-based model discover and leverage independent, sparsely interacting recurrent mechanisms to model dynamic environments?
• RQ2 Do RIMs improve generalization when a subset of underlying mechanisms changes between training and testing?
• RQ3 How does sparse activation and communication affect robustness to distribution shifts and novel objects/distractors?
• RQ4 Are RIMs effective as drop-in replacements for standard recurrent layers in reinforcement learning and vision tasks?

主要发现

模型	k_T	k_A	h_size	CE_Train	CE_Test
RIMs	6	4	600	0.00	0.00
RIMs	6	3	600	0.00	0.00
RIMs	6	2	600	0.00	0.00
RIMs	5	2	500	0.00	0.00
LSTM	-	-	300	0.00	4.32
LSTM	-	-	600	0.00	3.56
NTM	-	-	-	0.00	2.54
RMC	-	-	-	0.00	0.13
Transformers	-	-	-	0.00	0.54
“End of Table”	-	-	-	-	-
Sequential MNIST	-	-	-	-	-
RIMs	6	6	600	85.5	56.2
RIMs	6	5	600	88.3	43.1
RIMs	6	4	600	90.0	73.4
LSTM	-	-	300	86.8	42.3
LSTM	-	-	600	84.5	52.2
EntNet	-	-	-	89.2	52.4
RMC	-	-	-	89.58	54.23
DNC	-	-	-	87.2	44.1
Transformers	-	-	-	91.2	51.6

• RIMs specialize over temporal patterns, enabling some mechanisms to activate on distinct patterns and maintain performance under longer dormant phases.
• On the copying task, RIMs generalize to longer sequences (Test 200) where traditional models degrade, e.g., outperforming LSTM, NTM, and RMC in certain settings.
• In Sequential MNIST, RIMs are more robust to resolution changes (14x14 training to 19x19/24x24 testing) than LSTM and several baselines.
• RIMs improve robustness to novel distractors and partial observability in object-picking tasks, outperforming LSTM and competitive baselines.
• In PPO-based Atari experiments, replacing LSTM with RIMs yields substantial performance gains across games.
• Ablations show input attention, sparse activation, and inter-RIM communication are important for performance.