[Paper Review] On universal approximation and error bounds for Fourier Neural Operators

Nikola Kovachki, Samuel Lanthaler+1 more|CaltechAUTHORS (California Institute of Technology)|Jul 15, 2021

Model Reduction and Neural Networks30 references34 citations

TL;DR

The paper proves Fourier Neural Operators (FNOs) are universal for approximating continuous operators between function spaces, and provides explicit error bounds showing efficient approximation for PDE-related operators (e.g., Darcy type elliptic and incompressible Navier-Stokes).

ABSTRACT

Fourier neural operators (FNOs) have recently been proposed as an effective framework for learning operators that map between infinite-dimensional spaces. We prove that FNOs are universal, in the sense that they can approximate any continuous operator to desired accuracy. Moreover, we suggest a mechanism by which FNOs can approximate operators associated with PDEs efficiently. Explicit error bounds are derived to show that the size of the FNO, approximating operators associated with a Darcy type elliptic PDE and with the incompressible Navier-Stokes equations of fluid dynamics, only increases sub (log)-linearly in terms of the reciprocal of the error. Thus, FNOs are shown to efficiently approximate operators arising in a large class of PDEs.

Motivation & Objective

• Show that Fourier Neural Operators (FNOs) are universal for approximating continuous operators between function spaces.
• Derive explicit error bounds for FNOs when approximating PDE-related operators.
• Demonstrate mechanisms by which Ψ-FNOs approximate operators from PDEs efficiently, with polynomial/logarithmic scaling in error.
• Compare FNOs with alternative operator learning frameworks and discuss practical implications for PDE learning.

Proposed method

• Define the operator learning setting with inputs and outputs in Banach spaces and introduce Neural Operators as compositions of lifting, non-linear layers, and projection.
• Specialize to Fourier Neural Operators where kernels are translation-invariant and implemented via Fourier transforms.
• Introduce Ψ-FNOs which use discrete Fourier transforms and pseudo-spectral projections for efficient computation.
• Prove universal approximation of continuous operators by FNOs on compact subsets of Sobolev spaces (Theorem 2.5).
• Provide a mechanism to extend universality to Lipschitz domains via period extensions (Theorem 2.9).
• Discuss how non-linear lifting/projection can be accommodated and how error propagates through operator compositions.

Experimental results

Research questions

• RQ1 Can Fourier Neural Operators approximate a broad class of continuous operators between infinite-dimensional spaces?
• RQ2 What are the error bounds and network-size requirements for FNOs to approximate given operators to a specified accuracy?
• RQ3 Do Ψ-FNOs (pseudo-spectral implementations) achieve efficient approximation for PDE-derived operators?
• RQ4 How do FNOs compare to other operator-learning frameworks in terms of universality and efficiency?
• RQ5 Can the universality and efficiency results extend to domains with Lipschitz boundaries via period-extension techniques?

Key findings

• FNOs are universal: they can approximate any continuous operator from H^s to H^{s'} on compact subsets with arbitrary accuracy.
• In the worst case, the network size may grow exponentially with respect to the desired accuracy for general operators.
• For Ψ-FNOs approximating operators arising from Darcy-type elliptic equations and incompressible Navier–Stokes equations, the required network size scales polynomially (log-linearly) in the inverse of the error, indicating efficient approximation.
• Ψ-FNOs enable practical, efficient computation by leveraging discrete Fourier transforms and FFT-based instrumentation.
• The theoretical results provide the first rigorous justification for using FNOs in operator learning for PDEs, aligning with observed computational performance.