[论文解读] Discovering objects and their relations from entangled scene representations

David Raposo, Adam Santoro+4 more|arXiv (Cornell University)|Feb 16, 2017

Multimodal Machine Learning Applications被引用 73

一句话总结

关系网络（RNs）在场景中学习对象关系，具有置换不变性，能够分解缠结的输入；它们可以与记忆增强网络配对，以实现一次性关系学习。

ABSTRACT

Our world can be succinctly and compactly described as structured scenes of objects and relations. A typical room, for example, contains salient objects such as tables, chairs and books, and these objects typically relate to each other by their underlying causes and semantics. This gives rise to correlated features, such as position, function and shape. Humans exploit knowledge of objects and their relations for learning a wide spectrum of tasks, and more generally when learning the structure underlying observed data. In this work, we introduce relation networks (RNs) - a general purpose neural network architecture for object-relation reasoning. We show that RNs are capable of learning object relations from scene description data. Furthermore, we show that RNs can act as a bottleneck that induces the factorization of objects from entangled scene description inputs, and from distributed deep representations of scene images provided by a variational autoencoder. The model can also be used in conjunction with differentiable memory mechanisms for implicit relation discovery in one-shot learning tasks. Our results suggest that relation networks are a potentially powerful architecture for solving a variety of problems that require object relation reasoning.

研究动机与目标

• Motivate the need for reasoning about objects and relations in structured scenes.
• Propose a neural architecture (Relation Networks) that operates on pairs of objects with permutation invariance.
• Demonstrate RN capability to classify scenes based on relational structure.
• Show RN as a bottleneck to factorize entangled scene inputs into object-like representations.
• Demonstrate combination of RN with memory-augmented networks for one-shot relation learning.

提出的方法

• Define objects as feature vectors in a scene description matrix D (m objects by n features).
• Compute relations with a shared MLP gψ on all pairs of objects and aggregate via a commutative/associative function a (typically sum).
• Use a final function fφ to produce predictions from a, i.e., r̃ = fφ(Σij gψ(oi, oj)).
• Evaluate RN on supervised tasks where targets are adjacency matrices describing object relations.
• Demonstrate RN can induce factorization of objects from entangled inputs using a linear bottleneck layer or a VAE preprocessor.
• Combine RN with a Memory-Augmented Neural Network (MANN) to perform one-shot relation learning.

实验结果

研究问题

• RQ1 Can RNs learn and generalize object-relational structure from scene descriptions?
• RQ2 Can RNs infer object factorization and relations from entangled or pixel-based inputs?
• RQ3 Do RNs support one-shot learning when combined with memory modules?
• RQ4 How does RN performance compare to MLP baselines on relational tasks?
• RQ5 Can RN-mediated representations enable generalization to unseen relational graphs?

主要发现

• RNs outperform similarly sized MLPs on relational scene classification tasks and generalize to unseen classes.
• RNs can infer object relations from entangled inputs by learning a linear disentangler before the RN, revealing emergent object-factorized representations.
• A VAE-based perceptual pathway can feed latent codes into the RN, demonstrating RN compatibility with distributed image representations.
• RN-preprocessed MANNs achieve high one-shot relational classification within episodes, while MANNs with MLP preprocessors perform at chance.
• RN capabilities persist when used with memory and perceptual modules, indicating broad applicability for relational reasoning tasks.