[Paper Review] Object-Centric Learning with Slot Attention

Francesco Locatello, Dirk Weissenborn+6 more|arXiv (Cornell University)|Jun 26, 2020

Multimodal Machine Learning ApplicationsComputer Science89 references218 citations

TL;DR

The paper introduces Slot Attention, an iterative attention module that converts CNN perceptual features into a set of exchangeable slots that can bind to objects, enabling unsupervised object discovery and supervised set-based property prediction.

ABSTRACT

Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features. Yet, most deep learning approaches learn distributed representations that do not capture the compositional properties of natural scenes. In this paper, we present the Slot Attention module, an architectural component that interfaces with perceptual representations such as the output of a convolutional neural network and produces a set of task-dependent abstract representations which we call slots. These slots are exchangeable and can bind to any object in the input by specializing through a competitive procedure over multiple rounds of attention. We empirically demonstrate that Slot Attention can extract object-centric representations that enable generalization to unseen compositions when trained on unsupervised object discovery and supervised property prediction tasks.

Motivation & Objective

• Motivate learning object-centric representations to improve sample efficiency and generalization for visual scene understanding.
• Introduce Slot Attention as a differentiable interface between perceptual encoders and a set of slots.
• Demonstrate unsupervised object discovery with competitive performance and improved training efficiency.
• Showcase supervised set prediction where slots correspond to objects and can predict properties.
• Discuss generalization to unseen object compositions and numbers of objects.

Proposed method

• Present the Slot Attention module that maps N input feature vectors to K slots via iterative attention and a shared GRU-based update.
• Use dot-product attention with normalization over slots to create competition among slots for explaining input parts.
• Update slots with a GRU and optional residual MLP after each iteration, with LayerNorm for stable training.
• Initialize slots by sampling from a learnable Gaussian to allow test-time variability in slot counts.
• Apply the module as (i) an unsupervised object discovery encoder–decoder, and (ii) a set-prediction encoder for object properties.

Experimental results

Research questions

• RQ1 Can Slot Attention extract object-centric representations from perceptual inputs without supervision?
• RQ2 Does Slot Attention enable accurate unsupervised object discovery across multi-object datasets?
• RQ3 Can the learned slots support supervised property prediction for sets of objects?
• RQ4 How does Slot Attention generalize to more objects or different numbers of slots at test time?

Key findings

• Slot Attention achieves ARI scores competitive with or superior to state-of-the-art unsupervised object discovery methods on CLEVR6, Multi-dSprites, and Tetrominoes.
• On CLEVR6, ARI=98.8±0.3; Multi-dSprites ARI=91.3±0.3; Tetrominoes ARI=99.5±0.2 (with one outlier excluded).
• Compared to IODINE and MONet, Slot Attention is more memory-efficient and faster to train.
• For set prediction on CLEVR10, Slot Attention matches or outperforms the DSPN baseline in Average Precision and scales with more test-time iterations.
• Attention masks produced by Slot Attention can semantically segment objects, even without direct segmentation supervision.
• The method maintains strong performance when test-time slot count increases beyond training slots.