[Paper Review] Numerical posterior distribution error control and expected Bayes Factors in the bayesian Uncertainty Quantification of Inverse Problems

J. Andrés Christen, Marcos A. Capistrán+1 more|arXiv (Cornell University)|Jul 7, 2016

Probabilistic and Robust Engineering Design16 references3 citations

TL;DR

This paper proposes a practical error control framework for numerical solvers in Bayesian uncertainty quantification of inverse problems, ensuring the numerical posterior remains indistinguishable from the theoretical posterior. By bounding the global error of the solver using expected Bayes factors, the method enables computationally efficient, adaptive solvers that maintain posterior accuracy even with lower-resolution approximations.

ABSTRACT

In the bayesian analysis of Inverse Problems most relevant cases the forward maps (FM, or regressor function) are defined in terms of a system of (O, P)DE's with intractable solutions. These necessarily involve a numerical method to find approximate versions of such solutions and lead to a numerical/approximate posterior distribution. Recently several results have been published on the regularity conditions required on such numerical methods to ensure converge of the numerical to the theoretical posterior. However, more practical guidelines are needed to ensure a suitable working numerical posterior. ]Capistran2016] prove for ODE's that the Bayes Factor of the approximate vs the theoretical model tends to 1 in the same order as the numerical method order. In this work we generalize the latter paper in that we consider 1) also PDE's, 2) correlated observations, 3) practical guidelines in a multidimensional setting and 4) explore the use of expected Bayes Factors. This permits us to obtain bounds on the absolute global errors to be tolerated by the FM numerical solver, which we illustrate with some examples. Since the Bayes Factor is kept above 0.95 we expect that the resulting numerical posterior is basically indistinguishable from the theoretical posterior, even though we are using an approximate numerical FM. The method is illustrated with some examples using synthetic data.

Motivation & Objective

• To address the gap in practical guidelines for selecting numerical solver precision in Bayesian uncertainty quantification (UQ) of inverse problems.
• To extend prior theoretical convergence results to practical, real-world settings involving ODEs and PDEs with correlated observations.
• To provide a quantifiable bound on global solver error that ensures the numerical posterior is effectively indistinguishable from the theoretical posterior.
• To enable the use of adaptive, lower-resolution solvers without compromising posterior accuracy, reducing computational cost.
• To establish a framework for error control that is robust to observational noise and applicable to multidimensional, complex systems.

Proposed method

• Uses expected Bayes factors (BFs) as a metric to compare the numerical model (with approximate forward map) against the theoretical model (with exact forward map).
• Derives a bound on the global error of the numerical solver such that the expected BF remains close to 1, indicating negligible posterior difference.
• Applies the bound to both ODE and PDE systems, using adaptive mesh refinement to dynamically adjust solver resolution.
• Employs the twalk MCMC algorithm to sample from the posterior distributions under both high-resolution and adaptive solvers.
• Leverages regularity conditions and convergence order of numerical methods (e.g., Runge-Kutta, Discontinuous Galerkin) to relate solver error to posterior error.
• Uses synthetic data in examples to validate that the adaptive solver achieves similar posterior distributions with significantly reduced computational cost.

Experimental results

Research questions

• RQ1 How can numerical solver error be bounded to ensure the resulting numerical posterior is effectively indistinguishable from the theoretical posterior in Bayesian UQ?
• RQ2 What is the relationship between the convergence order of a numerical solver and the resulting Bayes factor between the numerical and theoretical models?
• RQ3 Can adaptive, lower-resolution solvers be used in Bayesian inverse problems without introducing detectable error in the posterior distribution?
• RQ4 How does observational noise influence the tolerable level of solver error in posterior inference?
• RQ5 To what extent can the proposed error control framework be generalized to PDE-based inverse problems with correlated observations?

Key findings

• The global error of the numerical solver must be bounded by a threshold proportional to the observational noise standard deviation σ and inversely proportional to the square root of the sample size, ensuring the expected Bayes factor remains near 1.
• In the ODE example, using an adaptive grid reduced CPU time by 60% (from 10.95h to 4.27h) while producing a posterior distribution indistinguishable from the high-resolution reference.
• In the PDE example (Burgers’ equation), the adaptive solver with error control produced posterior distributions nearly identical to those from a 512-point grid, confirming the method’s accuracy and efficiency.
• The expected Bayes factor converges to 1 at the same rate as the numerical method’s global error, validating the theoretical link between solver accuracy and posterior fidelity.
• The method enables the use of less precise, computationally cheaper solvers without compromising uncertainty quantification, provided the global error is kept within the derived bound.
• The framework is applicable to both ODEs and PDEs and extends to correlated observations, offering practical, workable guidelines for real-world Bayesian UQ.