[论文解读] A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

Lu Lu|arXiv (Cornell University)|Nov 9, 2021

Fluid Dynamics and Turbulent Flows参考文献 69被引用 431

一句话总结

本文对科学机器学习中学习非线性算子的DeepONet和傅里叶神经算子（FNO）进行了全面且公平的比较。研究提出了实用的扩展方法——POD-DeepONet、dFNO+和特征扩展，以提升复杂几何形状和噪声数据下的准确性和鲁棒性。主要发现是FNO对噪声极为敏感（例如，0.1%的噪声导致误差增加10,000倍），而DeepONet则保持鲁棒，表明FNO可能学习到不稳定的映射。

ABSTRACT

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

研究动机与目标

• 在多样且具有工业相关性的PDE基准上，对DeepONet和FNO进行系统且公平的比较。
• 通过新扩展方法（dFNO+、gFNO+）解决FNO在处理复杂几何形状和输入输出维度不匹配时的局限性。
• 通过基于POD的主干网络和特征扩展，提升DeepONet的训练速度和准确性。
• 评估模型在噪声数据下的鲁棒性以及泛化性能，特别是在存在测量不确定性的现实场景中。
• 在相同的正则性假设下进行理论比较，表明两种模型具有等价的误差估计。

提出的方法

• 提出dFNO+和gFNO+扩展方法，以处理复杂几何形状和不同维度的输入输出函数空间。
• 通过用训练数据中提取的本征正交分解模态替换主干网络，提出POD-DeepONet，以提升准确性。
• 对DeepONet应用特征扩展，以增强归纳偏置并加速收敛。
• 开发了一种利用快速傅里叶变换加速的DeepONet实现，使其计算成本与FNO相当。
• 使用16个多样化的基准测试——包括高超音速边界层、扑翼机翼和多孔介质流——涵盖稳态和非稳态PDE。
• 进行了理论分析，表明连续形式下的FNO是具有三角基函数和固定架构的DeepONet的特例。

实验结果

研究问题

• RQ1 在涉及复杂几何形状和噪声输入数据的实际条件下，DeepONet和FNO的表现如何？
• RQ2 FNO能否被扩展以处理不同维度函数空间之间以及不规则区域之间的映射？
• RQ3 与FNO相比，DeepONet在高噪声输入下是否仍保持鲁棒性？
• RQ4 在相同的正则性假设下，DeepONet和FNO的理论误差界是否一致？
• RQ5 像POD和特征扩展这样的实用扩展能否显著提升泛化能力和训练效率？

主要发现

• 在0.1%高斯噪声下的不稳定性波分析中，FNO的误差增加了10,000倍，导致其失效，而DeepONet的性能几乎未受影响。
• POD-DeepONet在仅使用10个训练样本的非稳态腔体流问题中实现了最低误差（0.18±0.02%），优于所有其他方法。
• 在使用90个训练样本的非稳态腔体流问题中，dFNO+和POD-DeepONet的相对L2误差分别为1.51%和1.78%，且应用了特征扩展。
• 仅使用10个训练样本时，带特征扩展的DeepONet在非稳态腔体流中实现了2.24%的相对误差，表现出强大的泛化能力。
• 理论分析证实，连续形式下的FNO是具有三角基函数和固定分支网络的DeepONet的特例。
• 在光滑、简单的设定下，两种模型的精度相当，但DeepONet在噪声和复杂几何场景中表现出更优的鲁棒性。