[Paper Review] Estimates on the generalization error of Physics Informed Neural Networks (PINNs) for approximating PDEs

Siddhartha Mishra, R. Molinaro|arXiv (Cornell University)|Jun 29, 2020

Model Reduction and Neural Networks30 citations

TL;DR

The paper provides upper bounds on the generalization error of PINNs for forward PDE problems, tying the error to training error, quadrature/sample size, and PDE stability, and illustrates the framework with several nonlinear PDE examples alongside numerical experiments.

ABSTRACT

Physics informed neural networks (PINNs) have recently been widely used for robust and accurate approximation of PDEs. We provide rigorous upper bounds on the generalization error of PINNs approximating solutions of the forward problem for PDEs. An abstract formalism is introduced and stability properties of the underlying PDE are leveraged to derive an estimate for the generalization error in terms of the training error and number of training samples. This abstract framework is illustrated with several examples of nonlinear PDEs. Numerical experiments, validating the proposed theory, are also presented.

Motivation & Objective

• Develop a general theoretical framework to bound the generalization error of PINNs solving forward PDEs.
• Relate the generalization error to training error, quadrature (sampling) error, and stability properties of the PDE.
• Provide concrete instances (parabolic, conservation laws, Euler equations) where the abstract bound applies and validate with numerical experiments.
• Clarify how the number and choice of training points affect error through quadrature analysis and stability constants.

Proposed method

• Formulate an abstract PDE setting with Banach spaces X and Y and a differential operator D mapping X* to Y*, including a stability bound (H3).
• Introduce an abstract PINN residual Rθ(uθ)=D(uθ)−f and a residual-based loss minimized over quadrature points.
• Derive a generalization error bound (Theorem 2.6) showing EG ≤ C_pde ET + C_pde C_quad^(1/p) N^(−α/p) under assumptions on stability and quadrature.
• Discuss quadrature rules (Gauss, low-discrepancy, Monte Carlo) and their convergence rates α and how they influence C_quad.
• Present an algorithmic framework (Algorithm 2.3) for training PINNs purely from the PDE residual without requiring solution data.
• Explain extensions to random training points and cumulative (average) generalization/training errors, including Lemma 2.10 (2.29).
• Illustrate the abstract framework with three concrete PDE examples: semi-linear parabolic equations, viscous scalar conservation laws, and incompressible Euler equations, plus numerical experiments.

Experimental results

Research questions

• RQ1 How can the generalization error of PINNs for forward PDE problems be bounded in terms of training error and sampling?
• RQ2 What roles do PDE stability and quadrature accuracy play in controlling the generalization error of PINNs?
• RQ3 How does the number and arrangement of training (collocation) points affect the derived error bounds in practical PDE settings?
• RQ4 Do the abstract bounds apply to nonlinear PDEs such as semi-linear parabolic equations, viscous conservation laws, and the incompressible Euler equations, and do numerical experiments corroborate the theory?

Key findings

• A generalization error bound is established: the total error is controlled by the training error and a quadrature-based term that scales with N and the quadrature rate α.
• The bound depends on stability constants tied to the PDE (C_pde) and a quadrature constant (C_quad) that reflect the residual and neural network details.
• The framework is instantiated for nonlinear PDEs, including semi-linear parabolic equations, viscous scalar conservation laws, and the incompressible Euler equations, with accompanying numerical experiments validating the theory.
• The analysis connects the PDE residual minimization in PINNs to provable control over generalization error, under reasonable regularity and stability assumptions.