[Paper Review] Interaction Networks for Learning about Objects, Relations and Physics

Peter Battaglia, Razvan Pascanu+3 more|arXiv (Cornell University)|Dec 1, 2016

Topic Modeling26 references576 citations

TL;DR

The paper introduces interaction networks, a graph-structured neural model that separately reasons about objects and their relations to predict dynamics and infer abstract properties like energy across diverse physical domains.

ABSTRACT

Reasoning about objects, relations, and physics is central to human intelligence, and a key goal of artificial intelligence. Here we introduce the interaction network, a model which can reason about how objects in complex systems interact, supporting dynamical predictions, as well as inferences about the abstract properties of the system. Our model takes graphs as input, performs object- and relation-centric reasoning in a way that is analogous to a simulation, and is implemented using deep neural networks. We evaluate its ability to reason about several challenging physical domains: n-body problems, rigid-body collision, and non-rigid dynamics. Our results show it can be trained to accurately simulate the physical trajectories of dozens of objects over thousands of time steps, estimate abstract quantities such as energy, and generalize automatically to systems with different numbers and configurations of objects and relations. Our interaction network implementation is the first general-purpose, learnable physics engine, and a powerful general framework for reasoning about object and relations in a wide variety of complex real-world domains.

Motivation & Objective

• Motivate and formalize reasoning about objects and relations in complex physical systems.
• Develop a general, learnable engine that separates object-centric and relation-centric reasoning.
• Demonstrate accurate multi-step trajectory prediction and abstract property inference.
• Show generalization to systems with different numbers and configurations of objects and relations.

Proposed method

• Represent systems as attributed directed multigraphs with objects as nodes and relations as edges.
• Use a relation model f_R to compute interaction effects e_k for each relation.
• Aggregate interaction effects per receiver object and feed them, with object state and external effects, into an object model f_O to predict next states.
• Train shared f_R and f_O across all relations and objects with gradient-based optimization.
• Optionally add an abstraction module phi_A to predict global properties (e.g., potential energy).
• Implement using MLPs with a marshalling function m and aggregation a that ensure commutativity/associativity for variable-sized graphs.

Experimental results

Research questions

• RQ1 Can an object- and relation-centric neural model learn to predict future states in complex physical systems?
• RQ2 To what extent can the model generalize to systems with different numbers and configurations of objects and relations?
• RQ3 Can the model infer abstract system properties such as potential energy from learned dynamics?
• RQ4 Does separating relation- and object-centric reasoning improve accuracy and generalization over baselines?
• RQ5 Is the approach scalable to multiple physical domains (n-body, rigid-body collisions, non-rigid dynamics)?

Key findings

• The model achieves predicting next-step velocities with much lower MSE than baselines across all domains.
• The interaction network generalizes to systems with fewer or more objects than seen during training, often outperforming models trained on smaller systems.
• Rollouts produced by the IN over thousands of steps visually resemble ground-truth physics, with coherent long-range dynamics.
• An abstract property estimator (for potential energy) using the IN substantially outperforms the MLP baseline (e.g., n-body and string domains).
• Regularization of interaction effects improves generalization and reduces drift in long rollouts.