[论文解读] On the Stability and Convergence of Physics Informed Neural Networks

Dimitrios Gazoulis, Ioannis Gkanis+1 more|arXiv (Cornell University)|Aug 10, 2023

Model Reduction and Neural NetworksPhysics and Astronomy被引用 3

一句话总结

本文通过利用能量泛函的强制性与$γ$-收敛，为物理信息神经网络（PINNs）的稳定性和收敛性建立了严格的数学框架。研究证明，稳定训练需要离散强制性，表明显式时间格式在缺乏严格CFL类约束时会导致不稳定，而隐式格式则在神经网络空间具备适当逼近性质时可确保收敛。

ABSTRACT

Physics Informed Neural Networks is a numerical method which uses neural networks to approximate solutions of partial differential equations. It has received a lot of attention and is currently used in numerous physical and engineering problems. The mathematical understanding of these methods is limited, and in particular, it seems that, a consistent notion of stability is missing. Towards addressing this issue we consider model problems of partial differential equations, namely linear elliptic and parabolic PDEs. Motivated by tools of nonlinear calculus of variations we systematically show that coercivity of the energies and associated compactness provide a consistent framework for stability. For time discrete training we show that if these properties fail to hold then methods may become unstable. Furthermore, using tools of $Γ$- convergence we provide new convergence results for weak solutions by only requiring that the neural network spaces are chosen to have suitable approximation properties. While our analysis is motivated by neural network-based approximation spaces, the framework developed here is applicable to any class of discrete functions satisfying the relevant approximation properties, and hence may serve as a foundation for the broader study of variational nonlinear PDE solvers.

研究动机与目标

• 为求解偏微分方程（PDEs）的物理信息神经网络（PINNs）缺乏一致的数学稳定性概念提供解决方案。
• 分析时间离散化训练下PINNs的稳定性，尤其对比显式与隐式时间格式。
• 利用$γ$-收敛理论，为线性椭圆与抛物型PDE的弱解提供收敛保证。
• 识别强制性与紧致性为确保PINN近似稳定与收敛的关键数学条件。
• 证明强制性失效会导致不稳定，尤其在显式时间离散化格式中。

提出的方法

• 将PINNs形式化为在神经网络空间上基于残差能量泛函的$L^2$-范数最小化问题。
• 应用非线性变分分析工具，特别是强制性与紧致性，为PINNs定义一种新的稳定性概念。
• 利用$γ$-收敛证明：当神经网络空间具备逼近性质时，PINN解可收敛至PDE的弱解。
• 通过隐式（IE）与显式（EE）欧拉型时间离散化分析时间离散PINN格式。
• 构造恢复序列以验证离散能量泛函的$γ$-极限，并建立最小化子的收敛性。
• 通过DeepXDE进行数值实验，比较在不同空间训练点与时间步长下显式与隐式时间格式的稳定性。

Figure 1: Explicit time discrete training. Left: time step $0.4:$ the approximate solution seems that diverge. Right: time step $0.01:$ with much smaller time step the approximate solution has stable behaviour.

实验结果

研究问题

• RQ1 能否为求解PDE的物理信息神经网络（PINNs）定义一致的稳定性概念？
• RQ2 能量泛函的强制性在确保PINN近似稳定性中起什么作用？
• RQ3 为何PINNs中的显式时间离散化格式会导致不稳定？在何种条件下可实现稳定？
• RQ4 在何种条件下，PINN能量泛函的最小化子会收敛至PDE的真实弱解？
• RQ5 神经网络空间的逼近性质如何影响PINN解的收敛性？

主要发现

• 能量泛函的强制性及其相关紧致性是PINNs中稳定性的充分必要条件，为稳定性提供了严格的数学基础。
• PINNs中的显式时间离散化无法满足强制性，导致不稳定与发散，尤其在时间步长过大或空间训练点增加时更为明显。
• 隐式时间离散化保持了强制性，确保了PINN近似向真实解的稳定收敛。
• 数值实验表明，显式PINNs在大时间步长（如$k=0.4$）下发散，但在小步长（如$k=0.01$）下趋于稳定，与理论分析一致。
• 当离散能量泛函$γ$-收敛且神经网络空间具备适当逼近性质时，PINN最小化子序列在$L^2(0,T;H^1(Ω))$中收敛至真实解$u$。
• 本研究证明，能量泛函的$γ$-收敛足以保证PINN解收敛至弱解，即使在离散设置中不假设一致强制性亦成立。

Figure 2: The approximations at times $t_{n}=n(0.2)$ , $n=1,2,\dots,$ are displayed with red and the initial condition with black. Left: Explicit time discrete training with time step $0.2$ and $16$ training points. The approximate solution seems that diverge. Left: Implicit time discrete training w