[Paper Review] Correcting model misspecification in physics-informed neural networks (PINNs)

Zongren Zou, Xuhui Meng+1 more|arXiv (Cornell University)|Oct 16, 2023

Model Reduction and Neural Networks4 citations

TL;DR

This paper proposes a method to correct model misspecification in physics-informed neural networks (PINNs) by introducing an additional deep neural network (DNN) to learn the discrepancy between a misspecified physical model and noisy or sparse observational data. The approach improves prediction accuracy in complex systems with incomplete physical understanding and integrates uncertainty quantification via B-PINNs or ensemble PINNs, demonstrating significant error reduction in reaction-diffusion and non-Newtonian flow problems.

ABSTRACT

Data-driven discovery of governing equations in computational science has emerged as a new paradigm for obtaining accurate physical models and as a possible alternative to theoretical derivations. The recently developed physics-informed neural networks (PINNs) have also been employed to learn governing equations given data across diverse scientific disciplines. Despite the effectiveness of PINNs for discovering governing equations, the physical models encoded in PINNs may be misspecified in complex systems as some of the physical processes may not be fully understood, leading to the poor accuracy of PINN predictions. In this work, we present a general approach to correct the misspecified physical models in PINNs for discovering governing equations, given some sparse and/or noisy data. Specifically, we first encode the assumed physical models, which may be misspecified, then employ other deep neural networks (DNNs) to model the discrepancy between the imperfect models and the observational data. Due to the expressivity of DNNs, the proposed method is capable of reducing the computational errors caused by the model misspecification and thus enables the applications of PINNs in complex systems where the physical processes are not exactly known. Furthermore, we utilize the Bayesian PINNs (B-PINNs) and/or ensemble PINNs to quantify uncertainties arising from noisy and/or gappy data in the discovered governing equations. A series of numerical examples including non-Newtonian channel and cavity flows demonstrate that the added DNNs are capable of correcting the model misspecification in PINNs and thus reduce the discrepancy between the physical models and the observational data. We envision that the proposed approach will extend the applications of PINNs for discovering governing equations in problems where the physico-chemical or biological processes are not well understood.

Motivation & Objective

• To address the challenge of model misspecification in PINNs when physical processes are incompletely understood.
• To improve prediction accuracy in PINNs by learning the discrepancy between imperfect physical models and observational data using an auxiliary DNN.
• To quantify uncertainties arising from noisy and gappy data in discovered governing equations.
• To enable the discovery of explicit governing equations by combining the method with symbolic regression.
• To extend the applicability of PINNs to complex systems where physico-chemical or biological processes are not fully known.

Proposed method

• Encode an assumed, potentially misspecified physical model into PINNs using automatic differentiation.
• Introduce an additional DNN to model the discrepancy between the imperfect model and observational data.
• Train the PINN with the discrepancy DNN to correct errors caused by model misspecification.
• Utilize Bayesian PINNs (B-PINNs) or ensemble PINNs to quantify uncertainty in predictions due to noisy or incomplete data.
• Combine the trained model with symbolic regression to extract explicit governing equations.
• Apply the framework to reaction-diffusion systems and non-Newtonian flows with known model misspecifications.

Experimental results

Research questions

• RQ1 Can a deep neural network effectively correct model misspecification in PINNs when the underlying physical model is incomplete or inaccurate?
• RQ2 How does the inclusion of a discrepancy DNN improve the accuracy of PINN predictions in systems with known model errors?
• RQ3 To what extent can B-PINNs or ensemble PINNs provide reliable uncertainty quantification for discovered physical models under noisy or gappy data?
• RQ4 Can the proposed method be seamlessly integrated with symbolic regression to derive interpretable governing equations?
• RQ5 Does the method maintain robustness and accuracy in complex physical systems such as non-Newtonian flows with misspecified viscosity?

Key findings

• The proposed method reduced the relative L2 error in velocity predictions from 7.79% to 1.33% and from 11.28% to 2.29% in the non-Newtonian cavity flow case.
• The error in force components (fx and fy) decreased from 3.9153×10⁻⁴ and 3.1466×10⁻⁴ to 7.8389×10⁻⁷ and 8.2283×10⁻⁷, respectively, demonstrating significant improvement.
• The discrepancy DNN effectively captured unmodeled physics, enabling accurate predictions even when the initial physical model was misspecified.
• B-PINNs and ensemble PINNs provided reasonable uncertainty bounds, reflecting model uncertainty due to data noise and gaps.
• The method successfully enabled symbolic regression to extract explicit governing equations after training, enhancing interpretability.
• The approach demonstrated robust performance across diverse systems, including reaction-diffusion and non-Newtonian flows, where physical models were intentionally misspecified.