[Paper Review] Physical Symmetries Embedded in Neural Networks

Marios Mattheakis, Pavlos Protopapas+3 more|arXiv (Cornell University)|Apr 18, 2019

Model Reduction and Neural Networks18 references61 citations

TL;DR

The paper embeds physical constraints into neural networks via hub neurons to enforce even/odd symmetry and energy conservation, including a symplectic NN for energy-conserving unsupervised DE solving. It demonstrates improved physical fidelity and robustness over standard networks.

ABSTRACT

Neural networks are a central technique in machine learning. Recent years have seen a wave of interest in applying neural networks to physical systems for which the governing dynamics are known and expressed through differential equations. Two fundamental challenges facing the development of neural networks in physics applications is their lack of interpretability and their physics-agnostic design. The focus of the present work is to embed physical constraints into the structure of the neural network to address the second fundamental challenge. By constraining tunable parameters (such as weights and biases) and adding special layers to the network, the desired constraints are guaranteed to be satisfied without the need for explicit regularization terms. This is demonstrated on upervised and unsupervised networks for two basic symmetries: even/odd symmetry of a function and energy conservation. In the supervised case, the network with embedded constraints is shown to perform well on regression problems while simultaneously obeying the desired constraints whereas a traditional network fits the data but violates the underlying constraints. Finally, a new unsupervised neural network is proposed that guarantees energy conservation through an embedded symplectic structure. The symplectic neural network is used to solve a system of energy-conserving differential equations and out-performs an unsupervised, non-symplectic neural network.

Motivation & Objective

• Motivate embedding physical constraints directly into neural network architectures to address physics-agnostic design and interpretability.
• Demonstrate hub neurons that enforce even/odd symmetry and energy conservation without explicit regularization.
• Show that embedded constraints improve predictive accuracy while preserving physical laws in supervised and unsupervised settings.
• Explore a symplectic neural network architecture that conserves energy when solving differential equations.

Proposed method

• Introduce hub neurons (hub layer) that enforce constraints through derived weights and biases to guarantee symmetry or conservation.
• Apply hub-layer design to regression tasks with even/odd symmetry under noisy data.
• Embed energy conservation in regression by a hub-layer ODE solver that corrects predictions to satisfy energy constraints.
• Develop a symplectic neural network by enforcing Hamiltonian structure via trial solutions and a constraint-based loss.
• Demonstrate symplectic NN on the Hénon-Heiles system and compare with a standard MLP and traditional solvers.

Experimental results

Research questions

• RQ1 Can hub neurons enforce physical symmetries (even/odd) in neural networks even when data are noisy?
• RQ2 Can embedded constraints guarantee energy conservation in regression tasks and unsupervised DE solving?
• RQ3 Does a symplectic neural network improve energy preservation and solution quality for energy-conserving dynamical systems?

Key findings

• Hub architecture enforces even/odd symmetry, reducing symmetry violations under noise and often speeding training.
• Energy-conserving regression improves alignment with physical energy, demonstrated on a harmonic oscillator.
• Symplectic NN solves energy-conserving DEs with lower training loss and better energy conservation than non-symplectic NNs.
• Symplectic NN delivers accurate trajectories and energy conservation in the Hénon–Heiles system, outperforming standard NN and standard ODE solvers in some metrics.