[Paper Review] Calibration of imperfect mathematical models by multiple sources of data with measurement bias

Mengyang Gu, K. R. Anderson|arXiv (Cornell University)|Oct 27, 2018

Soil Geostatistics and MappingEnvironmental Science42 references3 citations

TL;DR

This paper proposes the scaled Gaussian stochastic process (S-GaSP) to improve calibration of imperfect mathematical models by jointly modeling measurement bias and model discrepancy using multiple data sources. Unlike standard Gaussian stochastic processes (GaSP), S-GaSP uses a non-increasing scaling function that prioritizes solutions with smaller L₂ loss between model and reality, leading to consistent parameter estimation even with infinite data, as validated on a Kīlauea Volcano geophysical model using satellite interferograms.

ABSTRACT

Model calibration involves using experimental or field data to estimate the unknown parameters of a mathematical model. This task is complicated by discrepancy between the model and reality, and by possible bias in the data. We consider model calibration in the presence of both model discrepancy and measurement bias using multiple sources of data. Model discrepancy is often estimated using a Gaussian stochastic process (GaSP), but it has been observed in many studies that the calibrated mathematical model can be far from the reality. Here we show that modeling the discrepancy function via a GaSP often leads to an inconsistent estimation of the calibration parameters even if one has an infinite number of repeated experiments and infinite number of observations in a fixed input domain in each experiment. We introduce the scaled Gaussian stochastic process (S-GaSP) to model the discrepancy function. Unlike the GaSP, the S-GaSP utilizes a non-increasing scaling function which assigns more probability mass on the smaller $L_2$ loss between the mathematical model and reality, preventing the calibrated mathematical model from deviating too much from reality. We apply our technique to the calibration of a geophysical model of K\={\i}lauea Volcano, Hawai`i, using multiple radar satellite interferograms. We compare the use of models calibrated using multiple data sets simultaneously with results obtained using stacks (averages). We derive distributions for the maximum likelihood estimator and Bayesian inference, both implemented in the RobustCalibration package available on CRAN. Analysis of both simulated and real data confirm that our approach can identify the measurement bias and model discrepancy using multiple sources of data, and provide better estimates of model parameters.

Motivation & Objective

• To address the inconsistency of standard GaSP in calibrating imperfect models when both model discrepancy and measurement bias are present.
• To develop a method that ensures consistent parameter estimation under infinite data by prioritizing models close to reality via L₂ loss minimization.
• To enable joint calibration using multiple data sources, including repeated experiments and diverse observations, to better identify bias and discrepancy.
• To provide a statistically principled framework for uncertainty quantification in calibrated models through maximum likelihood and Bayesian inference.
• To demonstrate the method's effectiveness on real-world geophysical data, specifically Kīlauea Volcano deformation modeling using satellite interferograms.

Proposed method

• Introduce the scaled Gaussian stochastic process (S-GaSP), which modifies the prior covariance of the discrepancy function using a non-increasing scaling function of the L₂ loss between model and reality.
• Use the scaling function to assign higher prior probability to models with smaller L₂ loss, thereby discouraging large deviations from observed reality.
• Formulate a joint likelihood model that accounts for both measurement bias and model discrepancy, enabling simultaneous inference on calibration parameters.
• Derive the maximum likelihood estimator and posterior distributions under the S-GaSP framework for both frequentist and Bayesian inference.
• Implement the method in the RobustCalibration R package on CRAN for practical application and reproducibility.
• Validate the approach using both simulated data and real radar satellite interferograms from Kīlauea Volcano, comparing multi-source calibration with stacked (averaged) data.

Experimental results

Research questions

• RQ1 Can a Gaussian process model of discrepancy consistently estimate calibration parameters when both model discrepancy and measurement bias are present?
• RQ2 Does the use of a non-increasing scaling function in the S-GaSP improve consistency of parameter estimation compared to standard GaSP under infinite data?
• RQ3 Can multiple sources of data, including repeated measurements, jointly identify both measurement bias and model discrepancy more effectively than stacked data?
• RQ4 How does the S-GaSP framework perform in real-world geophysical modeling, such as for Kīlauea Volcano deformation?
• RQ5 What are the finite-sample properties of the maximum likelihood and Bayesian estimators under the S-GaSP model?

Key findings

• The standard GaSP leads to inconsistent calibration parameter estimation even with infinite data, due to insufficient regularization against large model discrepancies.
• The S-GaSP framework ensures consistent estimation by assigning higher prior probability to models with smaller L₂ loss between the model and reality.
• The method successfully identifies and corrects for measurement bias in both simulated and real data, improving model fidelity.
• Calibration using multiple data sources simultaneously outperforms stacking (averaging) of data, particularly in detecting bias and reducing uncertainty.
• The RobustCalibration R package enables practical implementation and statistical inference, with demonstrated performance on real satellite interferogram data from Kīlauea Volcano.
• Empirical analysis confirms that S-GaSP provides better parameter estimates and more reliable uncertainty quantification than standard GaSP or stacked data approaches.