[Paper Review] Finite element interpolated neural networks for solving forward and inverse problems

Santiago Badia, Wei Li+1 more|arXiv (Cornell University)|Jun 9, 2023

Model Reduction and Neural Networks50 references4 citations

TL;DR

This paper introduces Finite Element Interpolated Neural Networks (FEINNs), a novel framework that embeds neural networks into finite element spaces to solve forward and inverse PDE problems. By enforcing boundary conditions via FE interpolation and using variational residual minimization with preconditioned norms, FEINNs achieve superior convergence and generalization—outperforming both standard FEM and PINN-based methods, especially in smooth problems and complex geometries.

ABSTRACT

We propose a general framework for solving forward and inverse problems constrained by partial differential equations, where we interpolate neural networks onto finite element spaces to represent the (partial) unknowns. The framework overcomes the challenges related to the imposition of boundary conditions, the choice of collocation points in physics-informed neural networks, and the integration of variational physics-informed neural networks. A numerical experiment set confirms the framework's capability of handling various forward and inverse problems. In particular, the trained neural network generalises well for smooth problems, beating finite element solutions by some orders of magnitude. We finally propose an effective one-loop solver with an initial data fitting step (to obtain a cheap initialisation) to solve inverse problems.

Motivation & Objective

• Address the limitations of physics-informed neural networks (PINNs), such as poor convergence, boundary condition enforcement, and numerical integration errors.
• Overcome the challenges of training non-convex, nonlinear neural networks for PDEs by embedding them into conforming finite element spaces.
• Develop a robust, one-loop solver for inverse problems that avoids repeated forward problem solves by jointly learning state and parameters.
• Enable accurate and stable solution of PDE-constrained inverse problems with noisy or partial observations, without requiring regularization tuning.
• Demonstrate superior generalization and convergence rates compared to traditional FEM and existing PINN variants, especially in smooth regimes.

Proposed method

• Interpolate trained neural networks onto conforming finite element spaces using nodal interpolation, ensuring strong enforcement of Dirichlet boundary conditions via a standard FE offset function.
• Formulate the loss function as the L2 norm (or other norms) of the finite element residual, computed via exact integration over the mesh.
• Apply standard finite element preconditioners (e.g., geometric multigrid cycles) to stabilize the loss landscape and accelerate convergence in the h→0 limit.
• Use a three-step training strategy: (1) data fitting with no differential operators, (2) data fitting with residual terms, and (3) full coupled minimization to provide a high-quality initialization.
• Parametrize unknown model parameters (e.g., diffusion coefficients) via neural networks, which are also interpolated into FE spaces for consistent variational formulation.
• Combine data misfit and PDE residual penalty terms in the loss function, enabling joint optimization of state and parameters in a single training loop.

((a))

Experimental results

Research questions

• RQ1 Can neural networks embedded in finite element spaces outperform standard FEM and PINN-based methods in terms of convergence and generalization for smooth PDE solutions?
• RQ2 How does enforcing boundary conditions via FE interpolation compare to collocation-based PINN approaches in terms of accuracy and stability?
• RQ3 Can a one-loop training strategy with initial data fitting improve convergence and robustness in inverse problems without requiring adjoint solves?
• RQ4 How does the choice of residual norm and preconditioning affect training convergence and solution accuracy in FEINNs?
• RQ5 Can FEINNs handle complex geometries and non-homogeneous Dirichlet conditions more effectively than standard PINNs or IVPINNs?

Key findings

• FEINNs achieve significantly better generalization than standard FEM and non-interpolated PINN solutions, outperforming them by several orders of magnitude in smooth problems.
• For convection-diffusion-reaction problems, FEINNs converge to the FEM error level at higher mesh resolutions and polynomial orders where IVPINNs fail to converge.
• The three-step training strategy with initial data fitting leads to faster convergence and improved robustness, especially in inverse problems with partial or noisy observations.
• FEINNs demonstrate robustness without requiring regularization tuning, unlike adjoint-based NN methods that depend heavily on regularization for performance.
• In non-trivial geometries, FEINNs handle Dirichlet boundary conditions and complex domains more effectively than PINNs and IVPINNs, which struggle with boundary enforcement.
• Preconditioned residual norms (e.g., with GMG cycles) accelerate training and improve convergence, making the loss function well-posed in the h→0 limit.

((b))