[Paper Review] Symplectic Recurrent Neural Networks

Zhengdao Chen, Jianyu Zhang+2 more|arXiv (Cornell University)|Sep 29, 2019

Model Reduction and Neural NetworksPhysics and Astronomy24 references64 citations

TL;DR

SRNN learns Hamiltonian dynamics from trajectories using a neural Hamiltonian with symplectic integration, multi-step training, and initial state optimization, improving robustness to noise and handling stiff dynamics like rebounds.

ABSTRACT

We propose Symplectic Recurrent Neural Networks (SRNNs) as learning algorithms that capture the dynamics of physical systems from observed trajectories. An SRNN models the Hamiltonian function of the system by a neural network and furthermore leverages symplectic integration, multiple-step training and initial state optimization to address the challenging numerical issues associated with Hamiltonian systems. We show that SRNNs succeed reliably on complex and noisy Hamiltonian systems. We also show how to augment the SRNN integration scheme in order to handle stiff dynamical systems such as bouncing billiards.

Motivation & Objective

• Learn Hamiltonian dynamics directly from observed position and momentum trajectories.
• Improve robustness to observation noise via symplectic integration and recurrent training.
• Introduce initial state optimization to mitigate noise-induced bias.
• Demonstrate handling of stiff dynamics, including perfect rebound, via augmentation.
• Showcase performance on complex systems (spring-chain, three-body) and compare to baseline HNN/O-NET methods.

Proposed method

• Model the Hamiltonian as a neural network Hθ(p,q) whose derivatives give the dynamics (Hθ = Kθ1(p) + Vθ2(q)).
• Use a symplectic leapfrog integrator to propagate (p,q) and backpropagate through multiple time steps (multi-step training).
• Train with either ODE NET or Hamilitonian NET formulations, coupling training with integration to predict trajectories.
• Introduce Initial State Optimization (ISO): treat initial (p0,q0) as trainable per-trajectory variables optimized via loss.
• Augment leapfrog with a rebound module for stiff dynamics, using visual cues to model rebound direction and timing (n, α, γ).
• Demonstrate that training/testing with same integrator/time-step yields learned modified equations that compensate for discretization errors.

Experimental results

Research questions

• RQ1 Can SRNNs reliably learn complex Hamiltonian dynamics from noisy trajectory data?
• RQ2 Does using a symplectic (leapfrog) integrator improve stability and accuracy over Euler-based training?
• RQ3 Does recurrent (multi-step) training outperform single-step training under noise?
• RQ4 Can initial state optimization improve predictive accuracy under observation noise?
• RQ5 Can SRNN be extended to handle stiff dynamics such as perfect rebounds (billiards) via augmentation?

Key findings

• SRNN with recurrent (multi-step) training and leapfrog integration achieves the lowest predictive errors on the spring-chain and three-body systems compared with H-NET and O-NET baselines.
• Using leapfrog consistently in training and testing improves stability and accuracy over Euler-based schemes.
• Initial state optimization (ISO) yields the best predictions under noise, outperforming fixed-initial-state variants.
• SRNN-derived dynamics can compensate for discretization errors, sometimes outperforming numerically solving the true ODE with the same step size.
• Augmented SRNNs successfully learn perfect rebound behavior in a heavy billiard, outperforming baselines even when rebound timing is uncertain.