[Paper Review] A mixed formulation for physics-informed neural networks as a potential solver for engineering problems in heterogeneous domains: comparison with finite element method

Shahed Rezaei, Ali Harandi+3 more|arXiv (Cornell University)|Jun 27, 2022

Model Reduction and Neural NetworksPhysics and Astronomy63 references164 citations

TL;DR

This paper proposes a mixed formulation for physics-informed neural networks (PINNs) that separately predicts the primary variable and its spatial gradient via distinct neural networks, enforcing physical laws through strong-form PDEs and energy-based constraints using only first-order derivatives. The method achieves accuracy comparable to the finite element method (FEM) in solving elasticity and Poisson problems in heterogeneous domains, demonstrating PINNs' potential as a mesh-free, data-free solver for complex engineering problems.

ABSTRACT

Physics-informed neural networks (PINNs) are capable of finding the solution for a given boundary value problem. We employ several ideas from the finite element method (FEM) to enhance the performance of existing PINNs in engineering problems. The main contribution of the current work is to promote using the spatial gradient of the primary variable as an output from separated neural networks. Later on, the strong form which has a higher order of derivatives is applied to the spatial gradients of the primary variable as the physical constraint. In addition, the so-called energy form of the problem is applied to the primary variable as an additional constraint for training. The proposed approach only required up to first-order derivatives to construct the physical loss functions. We discuss why this point is beneficial through various comparisons between different models. The mixed formulation-based PINNs and FE methods share some similarities. While the former minimizes the PDE and its energy form at given collocation points utilizing a complex nonlinear interpolation through a neural network, the latter does the same at element nodes with the help of shape functions. We focus on heterogeneous solids to show the capability of deep learning for predicting the solution in a complex environment under different boundary conditions. The performance of the proposed PINN model is checked against the solution from FEM on two prototype problems: elasticity and the Poisson equation (steady-state diffusion problem). We concluded that by properly designing the network architecture in PINN, the deep learning model has the potential to solve the unknowns in a heterogeneous domain without any available initial data from other sources. Finally, discussions are provided on the combination of PINN and FEM for a fast and accurate design of composite materials in future developments.

Motivation & Objective

• To improve the performance of physics-informed neural networks (PINNs) in solving boundary value problems in heterogeneous domains.
• To reduce reliance on high-order derivatives in loss functions by introducing a mixed formulation with separate networks for primary variables and their gradients.
• To enhance physical consistency and convergence by incorporating both the strong form of the PDE and the energy functional as loss components.
• To compare the proposed mixed PINN approach directly with the finite element method (FEM) on benchmark elasticity and Poisson problems.
• To explore the feasibility of using PINNs as a mesh-free, unsupervised alternative to FEM for engineering problems without prior data.

Proposed method

• Train two separate neural networks: one for the primary variable (e.g., displacement or temperature), another for its spatial gradient (e.g., strain or flux).
• Enforce the strong form of the PDE (e.g., equilibrium or balance of forces) on the gradient network using automatic differentiation to compute second-order derivatives.
• Introduce the energy functional (e.g., total potential energy or dissipation) as an additional loss term to improve stability and accuracy.
• Construct the total loss function as a weighted sum of: (1) strong form residual, (2) boundary conditions (Dirichlet and Neumann), (3) energy functional, and (4) continuity constraints at interfaces.
• Use collocation points across the domain and boundaries to evaluate loss terms, with optimization via Adam optimizer.
• Apply the same network architecture (4 hidden layers, 20 neurons, tanh activation) and hyperparameters (learning rate = 0.001) for fair comparison with standard PINNs.

Experimental results

Research questions

• RQ1 Can a mixed formulation with separate networks for primary variables and their gradients improve PINN performance in heterogeneous solid mechanics problems?
• RQ2 Does using only first-order derivatives in the loss function lead to better convergence and accuracy compared to standard PINNs relying on higher-order derivatives?
• RQ3 How does the proposed mixed PINN compare in accuracy and convergence speed to the finite element method (FEM) for elasticity and steady-state diffusion problems?
• RQ4 To what extent does incorporating the energy functional as a loss term enhance the physical consistency and stability of the PINN solution?
• RQ5 Can the proposed PINN framework solve complex boundary value problems in heterogeneous domains without requiring labeled data or prior simulations?

Key findings

• The mixed PINN formulation, which uses separate networks for the primary variable and its gradient, achieves significantly lower mean absolute error (MAE) than standard PINNs when solving a second-order ODE.
• For the same number of collocation points and epochs, the first-order ODE formulation (using only first derivatives) produced lower MAE than the second-order ODE formulation, indicating improved stability and accuracy.
• Increasing the number of collocation points from 10 to 100 reduced the standard deviation of predictions and improved convergence, with the first-order ODE showing negligible variance beyond 100 points.
• The proposed mixed PINN achieved solution accuracy comparable to the finite element method (FEM) on both elasticity and Poisson equation problems in heterogeneous domains.
• The energy-based loss term contributed to improved physical consistency, especially in regions with material discontinuities or complex boundary conditions.
• The method demonstrated robust extrapolation behavior beyond the training domain, with predictions remaining close to the true solution even at x = 15 for a problem defined on [0,10].