[Paper Review] NeuralPDE: Automating Physics-Informed Neural Networks (PINNs) with Error Approximations

Kirill Zubov, Zoe McCarthy+12 more|arXiv (Cornell University)|Jul 19, 2021

Model Reduction and Neural NetworksPhysics and Astronomy49 citations

TL;DR

NeuralPDE presents a quadrature-based formulation that yields new loss functions with bounded error tolerances, enabling automated PINN workflows with symbolic code generation and GPU support.

ABSTRACT

Physics-informed neural networks (PINNs) are an increasingly powerful way to solve partial differential equations, generate digital twins, and create neural surrogates of physical models. In this manuscript we detail the inner workings of NeuralPDE.jl and show how a formulation structured around numerical quadrature gives rise to new loss functions which allow for adaptivity towards bounded error tolerances. We describe the various ways one can use the tool, detailing mathematical techniques like using extended loss functions for parameter estimation and operator discovery, to help potential users adopt these PINN-based techniques into their workflow. We showcase how NeuralPDE uses a purely symbolic formulation so that all of the underlying training code is generated from an abstract formulation, and show how to make use of GPUs and solve systems of PDEs. Afterwards we give a detailed performance analysis which showcases the trade-off between training techniques on a large set of PDEs. We end by focusing on a complex multiphysics example, the Doyle-Fuller-Newman (DFN) Model, and showcase how this PDE can be formulated and solved with NeuralPDE. Together this manuscript is meant to be a detailed and approachable technical report to help potential users of the technique quickly get a sense of the real-world performance trade-offs and use cases of the PINN techniques.

Motivation & Objective

• Provide a detailed technical report on NeuralPDE and how it automates PINNs.
• Explain how a quadrature-based formulation leads to adaptivity with error tolerances.
• Show how an extended loss formulation can assist parameter estimation and operator discovery.
• Demonstrate the framework's purely symbolic formulation and code generation from abstract formulations.

Proposed method

• Formulate PINN training loss using numerical quadrature to enable error-controlled adaptivity.
• Develop new loss functions derived from the quadrature-based formulation.
• Use a purely symbolic formulation so training code is generated from an abstract PDE formulation.
• Enable GPU utilization and solving of systems of PDEs within NeuralPDE.
• Showcase a complex multiphysics example (Doyle-Fuller-Newman model) within the framework.

Experimental results

Research questions

• RQ1 How can a quadrature-based formulation produce loss functions with bounded error tolerances for PINNs?
• RQ2 How does NeuralPDE support parameter estimation and operator discovery through extended loss formulations?
• RQ3 What are the performance trade-offs of different training techniques across a diverse set of PDEs?
• RQ4 How can a symbolic, abstract formulation be leveraged to automatically generate training code and enable GPU-accelerated PDE solving?

Key findings

• NeuralPDE yields new loss functions tied to numerical quadrature that enable adaptivity to error tolerances.
• The framework supports parameter estimation and operator discovery via extended loss formulations.
• Performance analysis compares training techniques across a large PDE set to reveal trade-offs.
• A complex multiphysics PDE, the Doyle-Fuller-Newman model, can be formulated and solved within NeuralPDE.