[Paper Review] Error estimates of residual minimization using neural networks for linear PDEs

Yeonjong Shin, Zhongqiang Zhang+1 more|arXiv (Cornell University)|Oct 15, 2020

Model Reduction and Neural Networks43 references34 citations

TL;DR

The paper develops an abstract convergence framework for neural-network-based residual minimization in linear PDEs, deriving a priori and a posteriori error estimates for both strong and weak formulations (PINN and variational PINN) and analyzing discrete vs continuous losses.

ABSTRACT

We propose an abstract framework for analyzing the convergence of least-squares methods based on residual minimization when feasible solutions are neural networks. With the norm relations and compactness arguments, we derive error estimates for both continuous and discrete formulations of residual minimization in strong and weak forms. The formulations cover recently developed physics-informed neural networks based on strong and variational formulations.

Motivation & Objective

• Introduce an abstract framework to analyze convergence of residual-minimization with neural networks for linear PDEs.
• Derive a priori and a posteriori error estimates for continuous and discrete loss formulations in strong and weak forms.
• Show convergence under suitable assumptions and connect to PINN and hp-VPINN variants.
• Validate theoretical assumptions with elliptic, advection–reaction, and fractional diffusion example problems.

Proposed method

• Formulate the linear problem A[u]=f, B[u]=g with norm relations C1 and C2 and establish existence/uniqueness under Assumption 3.
• Propose four loss functionals: discrete RM, continuous RM, discrete hp-VRM, and continuous hp-VRM.
• Prove error estimates and convergence results (Theorems 9 and 10) for continuous RM and relate them to universal approximation by neural networks (Assumption 2.3).
• Derive a priori and a posteriori error bounds (e.g., Theorem 9, equations (7)-(10)) and discuss discretization effects on convergence.
• Discuss compatibility/discretization assumptions and provide illustrative examples validating the framework.

Experimental results

Research questions

• RQ1 What conditions ensure convergence of neural-network-based residual minimization solutions to the true PDE solution?
• RQ2 How do continuous vs discrete and strong vs weak residuals affect error bounds and convergence?
• RQ3 What neural-network approximation properties are required (universal approximation, norm relations) for convergence?
• RQ4 How does discretization of the loss influence convergence, and what assumptions guarantee it?

Key findings

• Established an abstract framework linking graph-norm stability and universal approximation to convergence results.
• Derived a posteriori error estimate: ||u_N,n^τ - u*||_V ≤ C1^{-1} 2^{(p-1)/p} (J_τ(u_N,n^τ))^{1/p} for τ ≥ 1.
• Derived an a priori error bound involving the best neural-network approximation error plus data-approximation terms.
• Proved convergence of continuous RM: u_N,n^τ → u* in V as n → ∞ under the stated assumptions (Theorem 10).
• Highlighted the crucial role of discretization compatibility and demonstrated potential failure without appropriate assumptions (Example 4).