[Paper Review] Lagrangian Neural Networks

Miles Cranmer, Sam Greydanus+4 more|arXiv (Cornell University)|Mar 10, 2020

Computational Physics and Python Applications20 references66 citations

TL;DR

The paper introduces Lagrangian Neural Networks (LNNs) that learn arbitrary Lagrangians with neural networks, enabling energy-conserving dynamics without requiring canonical coordinates, and extends to graph-based and continuous systems via a Lagrangian Graph Network.

ABSTRACT

Accurate models of the world are built upon notions of its underlying symmetries. In physics, these symmetries correspond to conservation laws, such as for energy and momentum. Yet even though neural network models see increasing use in the physical sciences, they struggle to learn these symmetries. In this paper, we propose Lagrangian Neural Networks (LNNs), which can parameterize arbitrary Lagrangians using neural networks. In contrast to models that learn Hamiltonians, LNNs do not require canonical coordinates, and thus perform well in situations where canonical momenta are unknown or difficult to compute. Unlike previous approaches, our method does not restrict the functional form of learned energies and will produce energy-conserving models for a variety of tasks. We test our approach on a double pendulum and a relativistic particle, demonstrating energy conservation where a baseline approach incurs dissipation and modeling relativity without canonical coordinates where a Hamiltonian approach fails. Finally, we show how this model can be applied to graphs and continuous systems using a Lagrangian Graph Network, and demonstrate it on the 1D wave equation.

Motivation & Objective

• Motivate learning physical dynamics with stronger priors based on Lagrangian mechanics.
• Develop a neural framework that learns a Lagrangian without restricting kinetic energy form.
• Enable accurate, energy-conserving dynamics even when canonical coordinates are unknown or unavailable.
• Extend LNNs to graphs and continuous systems via a Lagrangian Graph Network for PDE-like tasks.

Proposed method

• Formulate dynamics from the Euler-Lagrange equation using a neural-network parameterized Lagrangian L(q, qdot).
• Compute accelerations by solving ddot{q} from a matrix equation involving Hessians of L and gradients, via ddot{q} = (nabla_{qdot} nabla_{qdot}^T L)^{-1} [nabla_q L - (nabla_q nabla_{qdot}^T L) dot{q}].
• Train by minimizing the discrepancy between predicted and true accelerations ddot{x}^L and ddot{x}^true.
• Allow non-canonical coordinates, unlike Hamiltonian approaches which require canonical momenta.
• Extend to Lagrangian Graph Networks by summing local Lagrangian densities over connected coordinate groups to model wave equations.
• Utilize JAX for efficient forward modeling and inverse Hessian computations; employ a novel initialization strategy tailored for LNNs.

Experimental results

Research questions

• RQ1 Can a neural network learn an arbitrary Lagrangian without enforcing canonical coordinates?
• RQ2 Do Lagrangian Neural Networks conserve energy more effectively than baseline models in long-time dynamics?
• RQ3 How does the Lagrangian formulation perform on non-canonical coordinate data compared to Hamiltonian-based approaches?
• RQ4 Can the LNN framework be extended to graph-structured or continuous systems via a Lagrangian Graph Network (LGN)?
• RQ5 What are practical training considerations (activation choice, initialization) for stable LNN learning?

Key findings

• LNNs conserve total energy much more accurately than a baseline neural network on the double pendulum, with energy discrepancy around 0.4% of max potential energy for LNNs versus 8% for baselines.
• On a relativistic particle with non-canonical coordinates, LNNs learn accurate dynamics whereas Hamiltonian networks fail without canonical coordinates.
• A Lagrangian Graph Network model can learn the 1D wave equation and conserve energy when aggregating Lagrangian densities over local grid neighborhoods.
• LNNs demonstrate the ability to learn non-trivial canonical momenta and dynamics in scenarios where Hamiltonian methods struggle due to coordinate constraints.