[论文解读] On the Role of Fixed Points of Dynamical Systems in Training Physics-Informed Neural Networks

Franz M. Rohrhofer, Stefan Posch+2 more|arXiv (Cornell University)|Mar 25, 2022

Model Reduction and Neural Networks被引用 15

一句话总结

本文 identifies fixed points in dynamical systems as a key cause of training instability in Physics-Informed Neural Networks (PINNs)，showing they create attractive local minima in the physics loss landscape that mislead optimization. By reducing the computational domain，the authors demonstrate smoother loss landscapes and improved training success，suggesting fixed points fundamentally shape PINN optimization complexity。

ABSTRACT

This paper empirically studies commonly observed training difficulties of Physics-Informed Neural Networks (PINNs) on dynamical systems. Our results indicate that fixed points which are inherent to these systems play a key role in the optimization of the in PINNs embedded physics loss function. We observe that the loss landscape exhibits local optima that are shaped by the presence of fixed points. We find that these local optima contribute to the complexity of the physics loss optimization which can explain common training difficulties and resulting nonphysical predictions. Under certain settings, e.g., initial conditions close to fixed points or long simulations times, we show that those optima can even become better than that of the desired solution.

研究动机与目标

• To investigate why Physics-Informed Neural Networks (PINNs) frequently fail to converge to the correct solution when training on dynamical systems。
• To examine the role of fixed points—equilibrium states inherent to dynamical systems—in shaping the loss landscape of PINNs。
• To determine how computational domain size and initial conditions near fixed points affect optimization success and loss landscape complexity。
• To evaluate whether fixed points can become better optima than the true solution，leading to nonphysical predictions。
• To explore the implications of these findings for existing PINN training techniques，such as loss weighting and collocation point resampling。

提出的方法

• Empirically analyze the physics loss landscape of PINNs trained on dynamical systems with known fixed points，using numerical simulations and visualization of loss surfaces。
• Vary the simulation time T to assess its impact on the complexity and structure of the loss landscape，particularly near fixed points。
• Use toy models with analytically known fixed points (e.g.，stable and unstable equilibria) to isolate the effect of fixed points from other training issues。
• Compare training outcomes under different initial conditions，especially those close to fixed points，to observe convergence to nonphysical solutions。
• Investigate the effect of domain reduction and collocation point sampling strategies on the prominence of fixed-point-induced minima in the loss landscape。
• Analyze the role of initial conditions and boundary constraints in preventing the network from converging to fixed-point solutions，even when they are attractive in the loss landscape。

实验结果

研究问题

• RQ1 Do fixed points in dynamical systems give rise to local minima in the physics loss function of PINNs？
• RQ2 Can fixed-point-induced minima become better than the true solution in terms of loss value，leading to nonphysical predictions？
• RQ3 How does reducing the computational domain (i.e.，simulation time T) affect the prominence of fixed-point-related minima in the loss landscape？
• RQ4 Why do PINNs trained on long-time simulations often fail to converge to the correct solution，even with proper initial and boundary conditions？
• RQ5 To what extent do existing PINN training techniques—such as loss weighting or collocation point reweighting—alleviate the issue of fixed-point attraction？

主要发现

• Fixed points in dynamical systems create local minima in the physics loss landscape of PINNs，which can trap optimization and lead to nonphysical predictions。
• When initial conditions are close to unstable fixed points，the loss landscape exhibits minima that are more attractive than the true solution，even if the true solution satisfies the governing equations。
• Reducing the simulation time T leads to smoother loss landscapes with less pronounced or absent fixed-point-induced minima，improving training success rates。
• The presence of fixed points explains why some PINN training failures persist despite proper initial and boundary condition enforcement，as these points can still dominate the loss landscape。
• Collocation point reweighting and resampling methods may indirectly reduce the effective domain size，thereby diminishing the influence of fixed-point minima，which aligns with the observed improvement in training。
• Loss weighting schemes may help overcome fixed-point attraction by increasing the penalty for violating initial/boundary conditions，suggesting that fixed points contribute to gradient imbalance in PINNs。