[Paper Review] Variational Physics-Informed Neural Networks For Solving Partial Differential Equations

Ehsan Kharazmi, Zhongqiang Zhang+1 more|arXiv (Cornell University)|Nov 27, 2019

Model Reduction and Neural Networks31 references197 citations

TL;DR

Introduces VPINN, a Petrov-Galerkin variational formulation of physics-informed neural networks, combining neural networks as trial space with Legendre test functions to solve PDEs more efficiently and accurately than PINNs.

ABSTRACT

Physics-informed neural networks (PINNs) [31] use automatic differentiation to solve partial differential equations (PDEs) by penalizing the PDE in the loss function at a random set of points in the domain of interest. Here, we develop a Petrov-Galerkin version of PINNs based on the nonlinear approximation of deep neural networks (DNNs) by selecting the {\em trial space} to be the space of neural networks and the {\em test space} to be the space of Legendre polynomials. We formulate the extit{variational residual} of the PDE using the DNN approximation by incorporating the variational form of the problem into the loss function of the network and construct a extit{variational physics-informed neural network} (VPINN). By integrating by parts the integrand in the variational form, we lower the order of the differential operators represented by the neural networks, hence effectively reducing the training cost in VPINNs while increasing their accuracy compared to PINNs that essentially employ delta test functions. For shallow networks with one hidden layer, we analytically obtain explicit forms of the extit{variational residual}. We demonstrate the performance of the new formulation for several examples that show clear advantages of VPINNs over PINNs in terms of both accuracy and speed.

Motivation & Objective

• Incorporate the variational (weak) form of PDEs into neural network-based solvers to improve accuracy and efficiency over strong-form PINNs.
• Reduce differential operator order via integration by parts to lower regularity requirements and training cost.
• Enable analytical tractability for shallow networks and facilitate domain decomposition and local learning.
• Demonstrate VPINN performance on representative PDEs and compare against PINNs.
• Provide analytical expressions for variational residuals in simple network settings to illuminate the method.”

Proposed method

• Formulate PDEs in a variational (weak) form and penalize the residual with a set of test functions within a neural network framework.
• Represent the solution with a deep neural network (trial space) while using a separate, linear test space (e.g., Legendre polynomials or sine functions).
• Construct a variational residual using inner products of the PDE operator with test functions and enforce it via a variational loss L^v.
• Use integration by parts to lower the differential order in the residual, reducing regularity requirements and training cost.
• For shallow networks, derive explicit analytical forms of the variational residuals; for deeper networks, apply numerical quadrature to evaluate integrals.
• Discuss multiple variational residual forms (Rk^(1), Rk^(2), Rk^(3)) and their impact on loss construction and boundary handling.”

Experimental results

Research questions

• RQ1 How does a variational (weak) formulation affect the training efficiency and accuracy of neural network PDE solvers compared to strong-form PINNs?
• RQ2 Can a Petrov-Galerkin VPINN framework reduce operator order and computational cost while maintaining or improving solution accuracy?
• RQ3 What are the analytical forms and numerical considerations for variational residuals in simple shallow VPINN setups?
• RQ4 How do test function choices and boundary treatments influence VPINN performance and stability?
• RQ5 What is the impact of domain decomposition and local test spaces on VPINN learning efficiency?

Key findings

• VPINNs show improved accuracy and speed over PINNs in the tested examples due to reduced operator order and fewer penalizing points.
• Analytical variational residuals can be derived for shallow networks with certain activations, illustrating the method’s tractability.
• Variational forms that incorporate boundary conditions via penalties can achieve high accuracy with properly chosen penalty parameters.
• Shallow VPINNs with sine activations and sine test functions yield explicit residual expressions and competitive errors in Burger’s equation tests.
• Boundary handling and optimization behavior (initialization, penalty parameter) significantly influence VPINN convergence and accuracy.”