[论文解读] When Do Extended Physics-Informed Neural Networks (XPINNs) Improve Generalization?

Zheyuan Hu, Ameya D. Jagtap+2 more|arXiv (Cornell University)|Sep 20, 2021

Model Reduction and Neural Networks被引用 12

一句话总结

本文提供了对扩展物理信息神经网络（XPINNs）在何种情况下优于标准PINNs实现更优泛化能力的首次理论分析。通过利用Barron空间和谱范数推导先验与后验泛化界，本文揭示了一个权衡关系：领域分解可降低每个子域的解复杂度（从而改善泛化），但同时限制了每个子网络的训练数据量（增加过拟合风险），当XPINNs在复杂度降低方面的优势超过数据稀缺性带来的负面影响时，其性能优于PINNs。

ABSTRACT

Physics-informed neural networks (PINNs) have become a popular choice for solving high-dimensional partial differential equations (PDEs) due to their excellent approximation power and generalization ability. Recently, Extended PINNs (XPINNs) based on domain decomposition methods have attracted considerable attention due to their effectiveness in modeling multiscale and multiphysics problems and their parallelization. However, theoretical understanding on their convergence and generalization properties remains unexplored. In this study, we take an initial step towards understanding how and when XPINNs outperform PINNs. Specifically, for general multi-layer PINNs and XPINNs, we first provide a prior generalization bound via the complexity of the target functions in the PDE problem, and a posterior generalization bound via the posterior matrix norms of the networks after optimization. Moreover, based on our bounds, we analyze the conditions under which XPINNs improve generalization. Concretely, our theory shows that the key building block of XPINN, namely the domain decomposition, introduces a tradeoff for generalization. On the one hand, XPINNs decompose the complex PDE solution into several simple parts, which decreases the complexity needed to learn each part and boosts generalization. On the other hand, decomposition leads to less training data being available in each subdomain, and hence such model is typically prone to overfitting and may become less generalizable. Empirically, we choose five PDEs to show when XPINNs perform better than, similar to, or worse than PINNs, hence demonstrating and justifying our new theory.

研究动机与目标

• 为了从理论上理解XPINNs在求解PDE时何时能优于标准PINNs实现更优泛化能力。
• 利用函数复杂度与网络容量度量，推导多层PINNs与XPINNs的泛化界。
• 识别XPINNs中子域内解复杂度降低与每个子网络训练数据受限之间的权衡关系。
• 通过解析示例与五个PDE的数值实验验证理论预测。

提出的方法

• 利用目标函数的Barron范数推导PINNs的先验泛化界，以衡量其复杂度，且无需额外假设。
• 通过网络权重重心的谱范数与(2,1)-范数建立后验泛化界，反映学习到的网络容量。
• 通过在每个子域上应用PINN边界并基于边界点分布进行加权平均，将PINN边界扩展至XPINNs。
• 利用定理3.1与定理3.2比较PINN与XPINN的泛化界，重点关注边界损失与数据稀缺性的影响。
• 采用解析PDE示例与五个基准PDE的数值实验验证理论预测。
• 通过评估子域复杂度降低与每个子网络训练数据减少之间的权衡，比较泛化性能。

Figure 2: Data visualization for the KdV experiment. Left: exact solution of the KdV equation. Right: Training points.

实验结果

研究问题

• RQ1 在何种条件下，XPINNs中的领域分解能实现优于标准PINNs的泛化性能？
• RQ2 子域内解复杂度的降低在多大程度上影响XPINNs的泛化性能？
• RQ3 XPINNs中每个子网络可用训练数据的减少在多大程度上损害泛化能力？
• RQ4 理论泛化界能否预测XPINNs在何时会优于、等于或劣于PINNs？
• RQ5 谱范数与Barron空间复杂度如何与PINNs和XPINNs的泛化行为相关联？

主要发现

• 当子域解复杂度的降低超过每个子网络训练数据受限的负面影响时，XPINNs可提升泛化性能。
• XPINNs的理论泛化界由子域Barron范数的加权和与子域边界点数量的平方根倒数之间的权衡决定。
• 在五个PDE上的实证结果证实，对于具有高多尺度或多物理场特征的问题，XPINNs在分解降低局部复杂度后优于PINNs。
• 对于解光滑且复杂度低的问题，PINNs通常因每个网络的有效训练数据更多而泛化性能更优。
• 基于谱范数与(2,1)-范数的后验泛化界在实践中能有效捕捉PINNs与XPINNs的泛化行为。
• 通过定理3.1与定理3.2推导的理论边界能成功预测不同PDE类型与分解策略下PINNs与XPINNs的相对性能。

Figure 3: Error visualization for the KdV experiment.