[Paper Review] Bayesian Calibration of Imperfect Computer Models using Physics-Informed Priors

Michail Spitieris, Ingelin Steinsland|arXiv (Cornell University)|Jan 17, 2022

Model Reduction and Neural Networks4 citations

TL;DR

This paper introduces a computationally efficient Bayesian calibration framework for imperfect computer models using physics-informed Gaussian process priors that encode differential equation structure in the covariance function. By combining physics-informed priors with model discrepancy modeling via Gaussian processes and Hamiltonian Monte Carlo inference, the method accurately recovers physical parameters and predictions even under model misspecification and biased data, with computational complexity reduced from O(N³) to O(N·m²) for big data.

ABSTRACT

We introduce a computational efficient data-driven framework suitable for quantifying the uncertainty in physical parameters and model formulation of computer models, represented by differential equations. We construct physics-informed priors, which are multi-output GP priors that encode the model's structure in the covariance function. This is extended into a fully Bayesian framework that quantifies the uncertainty of physical parameters and model predictions. Since physical models often are imperfect descriptions of the real process, we allow the model to deviate from the observed data by considering a discrepancy function. For inference, Hamiltonian Monte Carlo is used. Further, approximations for big data are developed that reduce the computational complexity from $\mathcal{O}(N^3)$ to $\mathcal{O}(N\cdot m^2),$ where $m \ll N.$ Our approach is demonstrated in simulation and real data case studies where the physics are described by time-dependent ODEs describe (cardiovascular models) and space-time dependent PDEs (heat equation). In the studies, it is shown that our modelling framework can recover the true parameters of the physical models in cases where 1) the reality is more complex than our modelling choice and 2) the data acquisition process is biased while also producing accurate predictions. Furthermore, it is demonstrated that our approach is computationally faster than traditional Bayesian calibration methods.

Motivation & Objective

• Address the challenge of parameter estimation in imperfect physical models described by differential equations, where model misspecification and data bias lead to systematic discrepancies.
• Develop a fully Bayesian framework that quantifies uncertainty in both physical parameters and model predictions, incorporating prior knowledge of the model structure.
• Introduce physics-informed Gaussian process priors that embed the differential equation structure directly into the covariance function to improve prior informativeness.
• Reduce computational cost for large datasets by introducing low-rank approximations that scale as O(N·m²) instead of O(N³), enabling application to big data.
• Demonstrate robustness in recovering true parameters and making accurate predictions when the real process is more complex than the model or when data acquisition is biased.

Proposed method

• Construct multi-output Gaussian process priors whose covariance functions are derived from the linear differential operators of the physical model (e.g., ODEs or PDEs), encoding the underlying physics directly.
• Use Hamiltonian Monte Carlo (HMC) for posterior inference on physical parameters and discrepancy functions, enabling efficient exploration of high-dimensional parameter spaces.
• Model systematic model discrepancy using a separate Gaussian process prior on the discrepancy function, allowing the model to deviate from observed data while preserving physical consistency.
• Integrate physics-informed priors with the Kennedy-O'Hagan (KOH) framework for Bayesian calibration, enabling joint inference on parameters and discrepancy with uncertainty quantification.
• Develop low-rank approximations using inducing points to reduce computational complexity from O(N³) to O(N·m²), where m ≪ N, by approximating the full GP using a subset of data points.
• Apply the framework to time-dependent ODEs (e.g., Windkessel model) and space-time PDEs (e.g., heat equation), using anisotropic squared exponential kernels for spatiotemporal correlation.

Experimental results

Research questions

• RQ1 Can physics-informed priors improve the accuracy and efficiency of Bayesian calibration in the presence of model-form uncertainty?
• RQ2 How well can the framework recover true physical parameters when the real process is more complex than the assumed model structure?
• RQ3 To what extent does the inclusion of a discrepancy function improve predictive accuracy under biased or noisy data?
• RQ4 Can the proposed low-rank approximation maintain accuracy while reducing computational cost for large datasets?
• RQ5 How does the integration of physics-informed priors with HMC and discrepancy modeling compare to traditional Bayesian calibration in terms of speed and robustness?

Key findings

• The method successfully recovers the true physical parameters of the Windkessel model even when the real hemodynamic process is more complex than the model structure.
• The framework produces accurate predictions for pressure and flow in the Windkessel model, even under biased data acquisition, due to the inclusion of a discrepancy function.
• The computational complexity is reduced from O(N³) to O(N·m²) using low-rank approximations, with m ≪ N, enabling scalability to large datasets.
• The physics-informed prior significantly improves prior informativeness, leading to faster convergence and more accurate posterior inference compared to non-informed priors.
• The method outperforms traditional Bayesian calibration in computational speed while maintaining or improving parameter recovery and predictive accuracy.
• The use of HMC enables efficient sampling from the joint posterior of parameters and discrepancy, ensuring robust uncertainty quantification across all model components.