[Paper Review] Discovering conservation laws from trajectories via machine learning

Seungwoong Ha, Hawoong Jeong|arXiv (Cornell University)|Feb 8, 2021

Model Reduction and Neural Networks31 references4 citations

TL;DR

This paper introduces ConservNet, a deep learning model that discovers conserved quantities from grouped trajectory data using a novel noise-variance loss function. By minimizing intra-group variance while resisting trivial constant solutions via a spreading loss, ConservNet successfully identifies invariants in simulated systems and real-world double pendulum data, even under noise and sparse conditions.

ABSTRACT

Invariants and conservation laws convey critical information about the underlying dynamics of a system, yet it is generally infeasible to find them from large-scale data without any prior knowledge or human insight. We propose ConservNet to achieve this goal, a neural network that spontaneously discovers a conserved quantity from grouped data where the members of each group share invariants, similar to a general experimental setting where trajectories from different trials are observed. As a neural network trained with a novel and intuitive loss function called noise-variance loss, ConservNet learns the hidden invariants in each group of multi-dimensional observables in a data-driven, end-to-end manner. Our model successfully discovers underlying invariants from the simulated systems having invariants as well as a real-world double pendulum trajectory. Since the model is robust to various noises and data conditions compared to baseline, our approach is directly applicable to experimental data for discovering hidden conservation laws and further, general relationships between variables.

Motivation & Objective

• To automate the discovery of conservation laws in dynamical systems from trajectory data without prior knowledge or human intervention.
• To address the challenge of identifying hidden invariants in noisy, sparse, and grouped experimental data where traditional methods fail.
• To develop a data-driven, end-to-end framework that directly extracts symbolic invariants from raw trajectories.
• To ensure robustness against noise, nuisance variables, and varying data conditions common in real-world experiments.
• To enable the discovery of multiple invariants, including those with large scale differences, through a scalable and generalizable approach.

Proposed method

• Proposes a neural network architecture, ConservNet, trained end-to-end on grouped trajectory data where each group shares a common invariant.
• Introduces a novel loss function called noise-variance loss that minimizes intra-group variance of model outputs to promote invariant learning.
• Incorporates a spreading term in the loss function that increases output variance under noisy input perturbations to prevent convergence to trivial constant solutions.
• Uses a noise vector $\bm{\varepsilon}_{ij}$ with bounded $L_2$-norm $R$ to simulate perturbations and control the strength of the spreading effect.
• Employs a hyperparameter $Q$ to regulate the target variance of the model output under noise, ensuring non-trivial invariant discovery.
• Applies a linear transformation $\hat{F}_{\theta} = aF_{\theta} + b$ to align model outputs with ground-truth invariants for evaluation.

Experimental results

Research questions

• RQ1 Can a deep neural network discover conserved quantities from grouped trajectory data without prior knowledge of the system’s dynamics?
• RQ2 How robust is the proposed method to noise, data sparsity, and varying group sizes in real-world experimental conditions?
• RQ3 Can the model distinguish between systems with and without conserved quantities, and detect the absence of invariants?
• RQ4 How does the model perform when multiple invariants coexist, especially when they differ significantly in scale?
• RQ5 To what extent can the noise-variance loss prevent convergence to trivial constant solutions while preserving non-trivial invariant learning?

Key findings

• ConservNet achieves Pearson correlation $\rho > 0.9$ across diverse data conditions, including extreme cases with only one long trajectory or 100 short trials.
• The model successfully discovers the angular momentum in the Kepler problem with $R^2 = 0.99$ when trained on $ (20,100) $ data condition.
• Even with a 10-fold scale difference between angular momentum and total energy, ConservNet shows strong correlation ($\rho \approx 0.9$) with the second invariant.
• The model performs robustly across different hyperparameters: $Q$ and $R$ values, and various noise norms ($L_1$, $L_2$, $L_\infty$), with $\rho > 0.98$ in all tested settings.
• In the absence of true invariants, ConservNet exhibits strong overfitting and high intra-group standard deviation, indicating reliable detection of non-conserved systems.
• The noise-variance loss effectively prevents trivial solutions, as confirmed by the model’s ability to maintain non-constant, meaningful outputs under perturbations.