[Paper Review] Learning Neural Hamiltonian Dynamics: A Methodological Overview

Zhijie Chen, Mingquan Feng+2 more|arXiv (Cornell University)|Feb 28, 2022

Model Reduction and Neural Networks4 citations

TL;DR

This paper presents a comprehensive survey of Hamiltonian neural networks (HNNs) that leverage physical laws—specifically energy conservation and symplectic geometry—to improve long-term prediction, interpretability, and data efficiency in learning dynamical systems. By learning the Hamiltonian function directly and enforcing symplectic integration, the method achieves superior accuracy and generalization over black-box models, especially in chaotic and high-dimensional systems.

ABSTRACT

The past few years have witnessed an increased interest in learning Hamiltonian dynamics in deep learning frameworks. As an inductive bias based on physical laws, Hamiltonian dynamics endow neural networks with accurate long-term prediction, interpretability, and data-efficient learning. However, Hamiltonian dynamics also bring energy conservation or dissipation assumptions on the input data and additional computational overhead. In this paper, we systematically survey recently proposed Hamiltonian neural network models, with a special emphasis on methodologies. In general, we discuss the major contributions of these models, and compare them in four overlapping directions: 1) generalized Hamiltonian system; 2) symplectic integration, 3) generalized input form, and 4) extended problem settings. We also provide an outlook of the fundamental challenges and emerging opportunities in this area.

Motivation & Objective

• To systematically review recent advances in Hamiltonian neural networks for modeling physical dynamics with improved accuracy and interpretability.
• To identify key methodological differences across HNN variants, focusing on generalized Hamiltonian systems, symplectic integration, input forms, and extended problem settings.
• To highlight the advantages of Hamiltonian-based learning in reducing parameters, enhancing generalization, and enabling few-shot learning in physical systems.
• To address limitations such as applicability to non-conservative systems and computational overhead, and to identify open challenges in chaotic and real-world dynamics.
• To provide a curated overview of open-source implementations and benchmark datasets for reproducibility and future research in the field.

Proposed method

• Proposes a framework that learns the Hamiltonian function H(q, p) as a neural network, from which the full dynamics are derived via Hamilton's equations: dq/dt = ∂H/∂p, dp/dt = -∂H/∂q.
• Employs symplectic integrators—particularly the leapfrog scheme—to preserve the geometric structure of Hamiltonian systems and prevent energy drift during long-term integration.
• Introduces generalized formulations such as dissipative HNNs (DHNN) and weak-form HNNs (GHNN) to extend applicability beyond conservative systems.
• Uses a loss function that enforces energy conservation by minimizing the time derivative of the Hamiltonian, either via pointwise or quadrature-based estimation.
• Applies adjoint methods and backpropagation through the integrator to train the Hamiltonian network end-to-end, ensuring differentiability and gradient flow.
• Employs techniques like Jacobian regularization and weak-form formulations to improve training stability and generalization in noisy or complex systems.

Experimental results

Research questions

• RQ1 How can Hamiltonian neural networks be systematically compared across key methodological dimensions such as generalized systems, symplectic integration, input forms, and problem settings?
• RQ2 What are the trade-offs between energy conservation, computational cost, and predictive accuracy in learning Hamiltonian dynamics?
• RQ3 In what ways do symplectic integrators improve long-term prediction fidelity compared to standard ODE solvers in chaotic systems?
• RQ4 How can Hamiltonian-based models be extended to non-conservative systems with energy dissipation while preserving physical consistency?
• RQ5 What are the fundamental limitations of current HNN frameworks in modeling real-world physical systems, especially those with noise, chaos, or non-separable dynamics?

Key findings

• Hamiltonian neural networks significantly reduce trainable parameters—down to thousands—compared to billions in black-box models, achieving comparable or better performance.
• Symplectic integrators, especially the leapfrog scheme, effectively suppress energy drift and improve long-term prediction accuracy in chaotic systems like the three-body and Hénon-Heiles problems.
• HNNs achieve high data efficiency and generalization, enabling few-shot learning and robust extrapolation beyond training distribution.
• The methodological extension to generalized Hamiltonian systems (e.g., GHNN, DHNN) allows modeling of non-conservative dynamics, broadening applicability beyond idealized physical systems.
• Despite advantages, HNNs face challenges in non-physical regimes (e.g., inelastic collisions), and training involves increased computational overhead due to Jacobian computation in the loss function.
• Open-source implementations and benchmark datasets (e.g., OpenAI Gym Pendulum, three-body system) are available, supporting reproducibility and further research in the field.