[论文解读] Physical Symmetries Embedded in Neural Networks

Marios Mattheakis, Pavlos Protopapas+3 more|arXiv (Cornell University)|Apr 18, 2019

Model Reduction and Neural Networks参考文献 18被引用 61

一句话总结

本论文通过在神经网络中嵌入物理约束，通过 hub neuron 来强制偶/奇对称性和能量守恒，并包括一个用于能量守恒的辛几何神经网络以解无监督的ODE。它展示了相比标准网络在物理保真度和鲁棒性方面的改进。

ABSTRACT

Neural networks are a central technique in machine learning. Recent years have seen a wave of interest in applying neural networks to physical systems for which the governing dynamics are known and expressed through differential equations. Two fundamental challenges facing the development of neural networks in physics applications is their lack of interpretability and their physics-agnostic design. The focus of the present work is to embed physical constraints into the structure of the neural network to address the second fundamental challenge. By constraining tunable parameters (such as weights and biases) and adding special layers to the network, the desired constraints are guaranteed to be satisfied without the need for explicit regularization terms. This is demonstrated on upervised and unsupervised networks for two basic symmetries: even/odd symmetry of a function and energy conservation. In the supervised case, the network with embedded constraints is shown to perform well on regression problems while simultaneously obeying the desired constraints whereas a traditional network fits the data but violates the underlying constraints. Finally, a new unsupervised neural network is proposed that guarantees energy conservation through an embedded symplectic structure. The symplectic neural network is used to solve a system of energy-conserving differential equations and out-performs an unsupervised, non-symplectic neural network.

研究动机与目标

• 直接将物理约束嵌入到神经网络架构中，以解决物理无关设计与可解释性的问题。
• 展示通过 hub neurons 强制偶/奇对称性和能量守恒而无需显式正则化。
• 表明嵌入式约束在监督和无监督设定下在保持物理定律的同时提升预测精度。
• 探索一种在求解微分方程时保持能量的辛几何神经网络架构。

提出的方法

• 引入 hub neurons（hub 层），通过推导得到的权重和偏置来强制约束，以保证对称性或守恒。
• 将 hub-layer 设计应用于在带噪数据下具有偶/奇对称性的回归任务。
• 通过 hub-layer 的 ODE 求解器将能量守恒嵌入回归中，从而纠正预测以满足能量约束。
• 通过试解形式和基于约束的损失来强制哈密顿结构，开发辛几何神经网络。
• 在 Hénon–Heiles 系统上演示辛几何神经网络，并与标准 MLP 和传统求解器进行比较。

实验结果

研究问题

• RQ1 在数据带噪的情况下，hub neurons 是否能够在神经网络中实现物理对称性（偶/奇）？
• RQ2 嵌入式约束是否能够在回归任务和无监督 DE 求解中保证能量守恒？
• RQ3 辛几何神经网络是否在能量守恒的动力系统中提升能量保持和解的质量？

主要发现

• Hub 架构强制偶/奇对称性，在噪声下减少对称性违规，且常常加速训练。
• 能量守恒回归在能量的一致性方面有所提升，在简谐振子的示例中得到证明。
• 辛几何神经网络在能量守恒的微分方程求解中，训练损失更低，能量守恒效果优于非辛几何神经网络。
• 辛几何神经网络在 Hénon–Heiles 系统中提供了准确轨迹和能量守恒，在某些指标上优于标准神经网络和标准 ODE 求解器。