[论文解读] Compositional Attention Networks for Machine Reasoning

Drew A. Hudson, Christopher D. Manning|arXiv (Cornell University)|Mar 8, 2018

Multimodal Machine Learning Applications参考文献 24被引用 132

一句话总结

Introduces the MAC network, a fully differentiable architecture that performs explicit, multi-step reasoning for visual question answering, achieving state-of-the-art on CLEVR and high data efficiency.

ABSTRACT

We present the MAC network, a novel fully differentiable neural network architecture, designed to facilitate explicit and expressive reasoning. MAC moves away from monolithic black-box neural architectures towards a design that encourages both transparency and versatility. The model approaches problems by decomposing them into a series of attention-based reasoning steps, each performed by a novel recurrent Memory, Attention, and Composition (MAC) cell that maintains a separation between control and memory. By stringing the cells together and imposing structural constraints that regulate their interaction, MAC effectively learns to perform iterative reasoning processes that are directly inferred from the data in an end-to-end approach. We demonstrate the model's strength, robustness and interpretability on the challenging CLEVR dataset for visual reasoning, achieving a new state-of-the-art 98.9% accuracy, halving the error rate of the previous best model. More importantly, we show that the model is computationally-efficient and data-efficient, in particular requiring 5x less data than existing models to achieve strong results.

研究动机与目标

• Motivate a neural architecture that supports explicit, structured reasoning rather than opaque end-to-end reasoning.
• Develop the MAC cell that separates control and memory to perform iterative reasoning steps.
• Demonstrate strong performance on CLEVR with high data-efficiency and interpretability.

提出的方法

• Propose a MAC cell with three units: control, read, and write, operating on dual states (control and memory).
• Use attention over the question words to guide each reasoning step, with a position-aware per-step question representation q_i.
• Employ a two-stage attention mechanism in the read unit over image regions guided by current control and memory.
• Integrate retrieved information into memory via the write unit, with optional self-attention over past memory and a memory gate to adapt reasoning length.
• Process inputs with a separate input unit: a biLSTM over the question and CNN-based image features to form a knowledge base K and question representation q.
• Output unit uses a classifier on the final memory state m_p and question to predict the answer.

实验结果

研究问题

• RQ1 Can a fully differentiable architecture learn explicit, multi-step reasoning without external program supervision?
• RQ2 Does separating control and memory with attention-based reasoning steps improve interpretability, data efficiency, and generalization for visual question answering?
• RQ3 How does the MAC architecture perform on counting and aggregation tasks within a VQA setting?
• RQ4 Is MAC robust to linguistic variation and capable of rapid learning from limited data?

主要发现

模型	CLEVR	计数	存在	比较	查询	比较	人类	人类	总体
MAC	98.9	97.1	99.5	99.1	99.5	99.5	57.4	81.5

• Achieves state-of-the-art CLEVR accuracy of 98.9% (significant improvement over prior models).
• Demonstrates strong performance on counting and numerical comparison tasks.
• Exhibits faster learning and greater data efficiency, requiring substantially less data to reach high accuracy.
• Shows robustness and better generalization, including on the CLEVR-Humans dataset after fine-tuning.
• Ablation studies confirm the importance of question attention, separation of control and memory, and explicit multi-step reasoning.
• Provides interpretable attention maps illustrating reasoning steps and transitive relations.