[Paper Review] Generic bounds on the approximation error for physics-informed (and) operator learning

Tim De Ryck, Siddhartha Mishra|arXiv (Cornell University)|May 23, 2022

Model Reduction and Neural Networks29 citations

TL;DR

The paper presents a general framework to derive rigorous bounds on approximation errors for PINNs and operator-learning architectures (DeepONets and FNOs), including physics-informed variants, and shows they can overcome the curse of dimensionality under certain smoothness assumptions.

ABSTRACT

We propose a very general framework for deriving rigorous bounds on the approximation error for physics-informed neural networks (PINNs) and operator learning architectures such as DeepONets and FNOs as well as for physics-informed operator learning. These bounds guarantee that PINNs and (physics-informed) DeepONets or FNOs will efficiently approximate the underlying solution or solution operator of generic partial differential equations (PDEs). Our framework utilizes existing neural network approximation results to obtain bounds on more involved learning architectures for PDEs. We illustrate the general framework by deriving the first rigorous bounds on the approximation error of physics-informed operator learning and by showing that PINNs (and physics-informed DeepONets and FNOs) mitigate the curse of dimensionality in approximating nonlinear parabolic PDEs.

Motivation & Objective

• Motivate and formalize a generic framework to bound approximation errors for PINNs and operator-learning architectures in PDE settings.
• Bridge error estimates across neural networks, space-time nets, and physics-informed variants to obtain unified guarantees.
• Derive first rigorous bounds for physics-informed operator learning and demonstrate dimension-independent convergence under smoothness assumptions.
• Illustrate how existing neural-network approximation results yield bounds for complex PDE-learning architectures.

Proposed method

• Establish a general Assumption framework (Assumption 3.1) linking solvable PDE solutions to efficiently approximable neural surrogates with controlled norms.
• Develop error transfer techniques showing how margin estimates for fixed-time NN approximations extend to space-time nets, PINNs, and physics-informed operator learners (Theorems 3.5–3.10).
• Utilize Taylor expansions for temporal discretization and finite-difference approximations for spatial derivatives to bound space-time approximation errors (Theorem 3.5).
• Leverage the known connection between Fourier neural operators (FNOs) and DeepONets to derive generic operator-learning bounds (Theorem 3.7 and Corollary 3.8).
• Impose a stability condition (Assumption 3.6) to obtain L2/L∞ error bounds for operators approximated by FNOs/DeepONets (Theorem 3.7).
• Provide a posteriori generalization bounds (Theorem 3.11) and show how training size interacts with architectural complexity.

Experimental results

Research questions

• RQ1 Can a unified framework yield rigorous, generic bounds on approximation errors for PINNs and operator-learning models (DeepONets and FNOs) across broad PDE classes?
• RQ2 Do physics-informed variants of operator learning (PINN-like DeepONets/FNOs) admit the same order of approximation as their data-driven counterparts, and can they mitigate the curse of dimensionality?
• RQ3 How can known neural-network approximation results be transferred to space-time networks and physics-informed operators to obtain efficient, dimension-independent convergence?
• RQ4 Under what conditions (smoothness, stability, and boundary assumptions) can one guarantee dimension-independent convergence rates for PINNs and physics-informed operator learning?
• RQ5 What are the generalization guarantees for these architectures in a learning-theory sense (a posteriori bounds)?

Key findings

• The authors present the first rigorous bounds for physics-informed operator learning (PINN-DeepONet/FNO variants).
• They prove space-time PINNs and physics-informed operators can achieve approximation errors that do not necessarily grow exponentially with dimension under certain regularity conditions (dimension-independence).
• They establish that if fixed-time neural networks approximate PDE solutions efficiently, then space-time networks and operator-learning surrogates can inherit efficient approximation properties (Theorem 3.5).
• A generic error bound for FNOs is derived, which via the known DeepONet-FNO connection yields corresponding bounds for DeepONets (Theorem 3.7 and Corollary 3.8).
• They provide an a posteriori generalization-error bound showing that, under standard conditions, expected generalization error can be controlled by training error and a complexity-dependent term (Theorem 3.11).
• Applications include showing PINNs overcome the curse of dimensionality for nonlinear parabolic PDEs (e.g., Allen–Cahn) and dimension-independent convergence for physics-informed operator learning (Section 4).