[论文解读] Data vs. Physics: The Apparent Pareto Front of Physics-Informed Neural Networks

Franz M. Rohrhofer, Stefan Posch+2 more|arXiv (Cornell University)|May 3, 2021

Model Reduction and Neural Networks参考文献 8被引用 15

一句话总结

本文研究了物理信息神经网络（PINNs）中数据保真度与物理约束之间的多目标优化权衡，表明由系统参数和尺度决定的Pareto前沿形状，对训练收敛性和精度具有决定性影响。研究发现，无量纲化可提高训练稳定性，且自适应损失加权优于自适应激活函数，能更有效地应对复杂的Pareto前沿。

ABSTRACT

Physics-informed neural networks (PINNs) have emerged as a promising deep learning method, capable of solving forward and inverse problems governed by differential equations. Despite their recent advance, it is widely acknowledged that PINNs are difficult to train and often require a careful tuning of loss weights when data and physics loss functions are combined by scalarization of a multi-objective (MO) problem. In this paper, we aim to understand how parameters of the physical system, such as characteristic length and time scales, the computational domain, and coefficients of differential equations affect MO optimization and the optimal choice of loss weights. Through a theoretical examination of where these system parameters appear in PINN training, we find that they effectively and individually scale the loss residuals, causing imbalances in MO optimization with certain choices of system parameters. The immediate effects of this are reflected in the apparent Pareto front, which we define as the set of loss values achievable with gradient-based training and visualize accordingly. We empirically verify that loss weights can be used successfully to compensate for the scaling of system parameters, and enable the selection of an optimal solution on the apparent Pareto front that aligns well with the physically valid solution. We further demonstrate that by altering the system parameterization, the apparent Pareto front can shift and exhibit locally convex parts, resulting in a wider range of loss weights for which gradient-based training becomes successful. This work explains the effects of system parameters on MO optimization in PINNs, and highlights the utility of proposed loss weighting schemes.

研究动机与目标

• 理解物理信息神经网络（PINNs）在多目标优化过程中出现收敛问题的根本原因。
• 研究系统参数和尺度如何影响PINN训练中Pareto前沿的形状。
• 评估自适应激活函数与自适应损失加权在改善PINN收敛性方面的有效性。
• 确定对物理系统进行恰当无量纲化是否能在无需手动超参数调优的情况下稳定PINN训练。
• 为未来PINN设计与超参数选择提供关于其内在优化景观的深入见解。

提出的方法

• 将PINN训练表述为一个平衡基于数据的损失与基于物理的损失（PDE残差）的多目标优化问题。
• 使用可视化技术构建损失景观，分析局部极小值与Pareto最优解。
• 系统性地改变系统参数（如长度、时间与材料尺度），研究其对Pareto前沿形状的影响。
• 应用最先进的自适应激活函数与自适应损失加权策略，评估其对优化轨迹的影响。
• 在多个测试案例中使用扩散方程与Navier-Stokes方程，评估方法在不同类型PDE上的泛化能力。
• 分析无量纲化对不同系统尺度下Pareto前沿及训练稳定性的影响。

实验结果

研究问题

• RQ1 系统参数与绝对尺度在多大程度上影响PINN训练中Pareto前沿的形状？
• RQ2 无量纲化在多大程度上改善了PINNs的收敛性与精度？
• RQ3 与标准PINNs相比，自适应激活函数在导航复杂Pareto前沿方面有多有效？
• RQ4 自适应损失加权方法是否能减少PINN训练中对手动超参数调优的依赖？
• RQ5 Pareto前沿的内在几何结构在决定PINNs中多目标优化成功与否方面起什么作用？

主要发现

• PINN训练中Pareto前沿的形状强烈受系统参数影响，尤其是计算域的绝对尺度。
• 无量纲化使Pareto前沿更优，通常呈现凸性，从而提升训练收敛性，并减少对人工损失加权的需求。
• 自适应损失加权方法能有效平衡不同系统尺度下的数据损失与物理损失，相较于基线PINNs表现出一致的改进。
• 自适应激活函数仅带来适度增益，且效果不如自适应损失加权，可能是因为其对全局优化景观的影响有限。
• PINNs中的多目标优化过程从根本上受限于Pareto前沿的内在几何结构，若缺乏适当的系统尺度调整，难以实现高精度。
• 即使采用自适应激活或损失加权等先进技术，收敛性仍对Pareto前沿的底层结构高度敏感，凸显了问题建模的重要性。