[Paper Review] A comprehensive and fair comparison of two neural operators (with practical extensions) based on FAIR data

Lu Lu|arXiv (Cornell University)|Nov 9, 2021

Fluid Dynamics and Turbulent Flows69 references431 citations

TL;DR

This paper presents a comprehensive, fair comparison of DeepONet and Fourier Neural Operator (FNO) for learning nonlinear operators in scientific machine learning. It introduces practical extensions—POD-DeepONet, dFNO+, and feature expansion—to enhance accuracy and robustness, especially for complex geometries and noisy data. The key finding is that FNO is highly sensitive to noise (e.g., 10,000× error increase with 0.1% noise), while DeepONet remains robust, indicating FNO may learn unstable mappings.

ABSTRACT

Neural operators can learn nonlinear mappings between function spaces and offer a new simulation paradigm for real-time prediction of complex dynamics for realistic diverse applications as well as for system identification in science and engineering. Herein, we investigate the performance of two neural operators, which have shown promising results so far, and we develop new practical extensions that will make them more accurate and robust and importantly more suitable for industrial-complexity applications. The first neural operator, DeepONet, was published in 2019 (Lu et al., 2019), and its original architecture was based on the universal approximation theorem of Chen & Chen (1995). The second one, named Fourier Neural Operator or FNO, was published in 2020, and it is based on parameterizing the integral kernel in the Fourier space. DeepONet is represented by a summation of products of neural networks (NNs), corresponding to the branch NN for the input function and the trunk NN for the output function; both NNs are general architectures, e.g., the branch NN can be replaced with a CNN or a ResNet. According to Kovachki et al. (2021), FNO in its continuous form can be viewed conceptually as a DeepONet with a specific architecture of the branch NN and a trunk NN represented by a trigonometric basis. In order to compare FNO with DeepONet computationally for realistic setups, we develop several extensions of FNO that can deal with complex geometric domains as well as mappings where the input and output function spaces are of different dimensions. We also develop an extended DeepONet with special features that provide inductive bias and accelerate training, and we present a faster implementation of DeepONet with cost comparable to the computational cost of FNO, which is based on the Fast Fourier Transform. Here we consider 16 different benchmarks to demonstrate the relative performance of the two neural operators, including instability wave analysis in hypersonic boundary layers, prediction of the vorticity field of a flapping airfoil, porous media simulations in complex-geometry domains, etc. We follow the guiding principles of FAIR (Findability, Accessibility, Interoperability, and Reusability) for scientific data management and stewardship. The performance of DeepONet and FNO is comparable for relatively simple settings, but for complex geometries the performance of FNO deteriorates greatly. We also compare theoretically the two neural operators and obtain similar error estimates for DeepONet and FNO under the same regularity assumptions.

Motivation & Objective

• To conduct a systematic, fair comparison of DeepONet and FNO across diverse, industrially relevant PDE benchmarks.
• To address limitations of FNO in handling complex geometries and mismatched input-output dimensions through new extensions (dFNO+, gFNO+).
• To improve DeepONet’s training speed and accuracy using POD-based trunk networks and feature expansion.
• To evaluate robustness to noisy data and generalization performance, especially in real-world scenarios with measurement uncertainty.
• To provide a theoretical comparison under the same regularity assumptions, showing equivalent error estimates for both models.

Proposed method

• Proposed dFNO+ and gFNO+ extensions to handle complex geometries and different input-output function space dimensions.
• Introduced POD-DeepONet by replacing the trunk network with proper orthogonal decomposition modes from training data for improved accuracy.
• Applied feature expansion to DeepONet to enhance inductive bias and accelerate convergence.
• Developed a faster DeepONet implementation using fast Fourier transforms to match FNO’s computational cost.
• Used 16 diverse benchmarks—including hypersonic boundary layers, flapping airfoils, and porous media flows—across steady and unsteady PDEs.
• Performed theoretical analysis showing FNO in continuous form is a special case of DeepONet with trigonometric basis and fixed architecture.

Experimental results

Research questions

• RQ1 How do DeepONet and FNO perform under realistic conditions involving complex geometry and noisy input data?
• RQ2 Can FNO be extended to handle mappings between function spaces of different dimensions and irregular domains?
• RQ3 Does DeepONet maintain robustness under high levels of input noise compared to FNO?
• RQ4 What theoretical error bounds are shared between DeepONet and FNO under identical regularity assumptions?
• RQ5 Can practical extensions like POD and feature expansion significantly improve generalization and training efficiency?

Key findings

• For instability wave analysis with 0.1% Gaussian noise, FNO’s error increased 10,000-fold, rendering it ineffective, while DeepONet showed negligible performance degradation.
• POD-DeepONet achieved the lowest error (0.18±0.02%) in unsteady cavity flow with 10 training samples, outperforming all other methods.
• In unsteady cavity flow with 90 training samples, dFNO+ and POD-DeepONet achieved relative L2 errors of 1.51% and 1.78%, respectively, with feature expansion.
• DeepONet with feature expansion achieved 2.24% relative error in unsteady cavity flow with only 10 training samples, demonstrating strong generalization.
• Theoretical analysis confirmed that FNO in continuous form is a special case of DeepONet with a trigonometric basis and fixed branch network.
• Both models exhibited similar accuracy in smooth, simple settings, but DeepONet showed superior robustness in noisy and complex-geometry scenarios.