[Paper Review] A Compositional Object-Based Approach to Learning Physical Dynamics

Michael B. Chang, Tomer Ullman+2 more|arXiv (Cornell University)|Dec 1, 2016

Computational Physics and Python Applications169 citations

TL;DR

The paper introduces the Neural Physics Engine (NPE), a differentiable, object-based physics simulator using pairwise interactions that generalizes to variable object counts and scene configurations and can infer latent object properties like mass.

ABSTRACT

We present the Neural Physics Engine (NPE), a framework for learning simulators of intuitive physics that naturally generalize across variable object count and different scene configurations. We propose a factorization of a physical scene into composable object-based representations and a neural network architecture whose compositional structure factorizes object dynamics into pairwise interactions. Like a symbolic physics engine, the NPE is endowed with generic notions of objects and their interactions; realized as a neural network, it can be trained via stochastic gradient descent to adapt to specific object properties and dynamics of different worlds. We evaluate the efficacy of our approach on simple rigid body dynamics in two-dimensional worlds. By comparing to less structured architectures, we show that the NPE's compositional representation of the structure in physical interactions improves its ability to predict movement, generalize across variable object count and different scene configurations, and infer latent properties of objects such as mass.

Motivation & Objective

• Motivate and design a prior on physics that generalizes across scenes with varying numbers of objects and configurations.
• Propose a differentiable, object-based physics engine that combines symbolic structure with gradient-based learning.
• Demonstrate that a compositional, pairwise-interaction architecture improves generalization and latent property inference.
• Evaluate the approach on two-dimensional World with balls and obstacles to test prediction, generalization, and mass inference.

Proposed method

• Factorize a physical scene into object-based representations with extrinsic/intrinsic/global properties.
• Use a neighborhood mask to select context objects for each focus object, implementing a broad phase-like context selection.
• Model the focus object's velocity as a sum of pairwise interactions encoded by a pair encoder and decoded to velocity change.
• Predict velocity change (Δv) and update position via a first-order velocity integration, keeping acceleration implicit through recent timesteps.
• Train the network end-to-end with stochastic gradient descent using observed trajectories from matter-js based simulations.
• Compare NPE to baselines (No-Pairwise NP and LSTM) to isolate the benefits of pairwise factorization and compositionality.

Experimental results

Research questions

• RQ1 Can the NPE accurately predict object dynamics in 2D ball-and-obstacle worlds?
• RQ2 Does the NPE generalize to scenes with more objects than seen during training?
• RQ3 Can the NPE infer latent object properties such as mass from dynamics?
• RQ4 What is the impact of the context neighborhood mask on predictive accuracy and generalization?
• RQ5 How does the NPE handle different scene configurations (wall geometries, obstacles) while maintaining performance?

Key findings

• The NPE consistently outperforms NP and LSTM baselines in velocity prediction and generalization tasks, especially with 6–8 objects during testing.
• Predictive accuracy and generalization for the NPE improve with training, while NP and LSTM plateau early.
• The NPE achieves about 90% accuracy in inferring the focused object's mass from observed collisions.
• The neighborhood mask substantially improves performance (roughly an order of magnitude in velocity prediction loss) and enables constant-time context handling (O(1)) per focus object.
• The NPE generalizes robustly to complex scene configurations (walls and internal obstacles) not seen during training, outperforming baselines as scene complexity increases.