[Paper Review] Hamiltonian Graph Networks with ODE Integrators

Álvaro Sánchez‐González, Victor Bapst+2 more|arXiv (Cornell University)|Sep 27, 2019

Topic ModelingComputer Science11 references75 citations

TL;DR

The paper combines graph networks with differentiable ODE integrators and a Hamiltonian-based internal representation to improve predictive and energy accuracy in learned physical dynamics and enable generalization to unseen time-steps and integrator orders.

ABSTRACT

We introduce an approach for imposing physically informed inductive biases in learned simulation models. We combine graph networks with a differentiable ordinary differential equation integrator as a mechanism for predicting future states, and a Hamiltonian as an internal representation. We find that our approach outperforms baselines without these biases in terms of predictive accuracy, energy accuracy, and zero-shot generalization to time-step sizes and integrator orders not experienced during training. This advances the state-of-the-art of learned simulation, and in principle is applicable beyond physical domains.

Motivation & Objective

• Introduce physically informed inductive biases by coupling graph networks with a differentiable ODE integrator for predicting future states.
• Impose Hamiltonian mechanics as an internal representation to enhance energy conservation.
• Demonstrate improved predictive accuracy and zero-shot generalization to novel time-steps and integrator orders.

Proposed method

• Represent particle systems as graphs and use a graph network to process state information.
• Use a differentiable Runge-Kutta integrator to simulate ODE dynamics with a learned time-derivative model.
• Define three model variants: OGN (ODE Graph Network) that learns time derivatives, and HOGN (Hamiltonian ODE Graph Network) that computes a Hamiltonian via a GN and derives derivatives from its gradients.
• HOGN computes H(q,p) using a GN over global features and uses ∂H/∂p and -∂H/∂q as the time derivatives to feed the integrator.
• Training compares DeltaGN (direct state change prediction) with OGN and HOGN across RK1–RK4 and, in supplementary material, symplectic integrators.
• Evaluation metrics include rollout error (position), energy error (conservation) and zero-shot time-step/generalization performance.

Experimental results

Research questions

• RQ1 Can incorporating Hamiltonian structure and an ODE integrator bias into graph networks improve long-horizon prediction for physical systems?
• RQ2 Does the Hamiltonian inductive bias yield better energy conservation and generalization across time steps and integrator orders compared to non-Hamiltonian baselines?
• RQ3 How do different integrators (RK1–RK4, symplectic variants) interact with OGN and HOGN in terms of accuracy and generalization?
• RQ4 To what extent can learned models generalize to time-steps and integrators not seen during training?

Key findings

• HOGN achieved the highest predictive accuracy in 20-step rollout tests with RK4.
• Energy error was lower for OGN and HOGN than the DeltaGN baseline, with HOGN matching true Hamiltonian behavior under certain conditions.
• Generalization to unseen time-steps and to higher-order integrators was stronger for HOGN and OGN than for DeltaGN, with HOGN often matching or closely approximating true Hamiltonian performance when trained with RK4.
• HOGN and OGN showed better zero-shot generalization to time-step sizes not experienced during training compared to DeltaGN.
• Using higher-order integrators during training (e.g., RK4) benefited Hamiltonian-based models more, particularly in energy conservation and cross-integrator generalization.
• Supplementary results indicate symplectic integrators can yield different generalization behaviors, with higher-order symplectic training sometimes improving energy conservation but not universally.