[Paper Review] Adaptive particle-based approximations of the Gibbs posterior for inverse problems

Zilong Zou, Sayan Mukherjee+2 more|arXiv (Cornell University)|Jul 2, 2019

Probabilistic and Robust Engineering Design35 references4 citations

TL;DR

This paper proposes an adaptive particle-based sequential Monte Carlo (SMC) method with local reduced basis (RB) surrogates to efficiently approximate the Gibbs posterior in inverse problems governed by PDEs. By leveraging a loss function instead of a likelihood, the method enables robust inference when the data mechanism is unknown, and achieves significant computational savings through locally adaptive, high-fidelity RB approximations of the loss function during SMC evolution.

ABSTRACT

In this work, we adopt a general framework based on the Gibbs posterior to update belief distributions for inverse problems governed by partial differential equations (PDEs). The Gibbs posterior formulation is a generalization of standard Bayesian inference that only relies on a loss function connecting the unknown parameters to the data. It is particularly useful when the true data generating mechanism (or noise distribution) is unknown or difficult to specify. The Gibbs posterior coincides with Bayesian updating when a true likelihood function is known and the loss function corresponds to the negative log-likelihood, yet provides subjective inference in more general settings. We employ a sequential Monte Carlo (SMC) approach to approximate the Gibbs posterior using particles. To manage the computational cost of propagating increasing numbers of particles through the loss function, we employ a recently developed local reduced basis method to build an efficient surrogate loss function that is used in the Gibbs update formula in place of the true loss. We derive error bounds for our approximation and propose an adaptive approach to construct the surrogate model in an efficient manner. We demonstrate the efficiency of our approach through several numerical examples.

Motivation & Objective

• Address the high computational cost of Bayesian inference in PDE-constrained inverse problems where likelihood evaluation is expensive.
• Overcome the inefficiency of globally accurate surrogate models by focusing on posterior-supported regions through adaptive sampling.
• Develop a framework that combines the flexibility of the Gibbs posterior—relying only on a loss function—with efficient particle-based inference.
• Enable scalable uncertainty quantification in inverse problems with unknown or complex noise models by avoiding explicit likelihood specification.
• Achieve computational efficiency through sequential, adaptive construction of local reduced basis models tailored to the evolving posterior support.

Proposed method

• Employ a sequential Monte Carlo (SMC) framework to iteratively evolve weighted particles approximating the Gibbs posterior distribution.
• Use a loss function (e.g., l2 loss) to define the Gibbs posterior, avoiding the need for a known likelihood or noise model.
• Construct a local reduced basis (RB) surrogate model for the loss function that is updated adaptively at each SMC step based on particle locations.
• Adapt the local RB model using a threshold-based refinement strategy, where the error tolerance is set to 5% of the standard deviation of current loss values.
• Use the local RB surrogate in place of the full PDE-based loss evaluation during SMC updates, drastically reducing per-particle cost.
• Ensure convergence and accuracy by deriving theoretical error bounds for the RB approximation within the SMC framework.

Experimental results

Research questions

• RQ1 Can adaptive local reduced basis models significantly reduce the computational cost of approximating the Gibbs posterior in PDE-constrained inverse problems?
• RQ2 How does the performance of the proposed SMC-RB method compare to standard MCMC in terms of accuracy and efficiency for inverse problems with unknown noise models?
• RQ3 To what extent does the adaptive refinement of local RB models improve accuracy on the posterior support compared to global surrogates?
• RQ4 How does the number of local RB atoms scale with noise level and parameter dimensionality in complex inverse problems?
• RQ5 Can the proposed method maintain consistency with MCMC-based inference while achieving orders-of-magnitude speedups?

Key findings

• The proposed method achieves computational savings by focusing local RB surrogate construction on the posterior-supported region, avoiding global accuracy requirements.
• For the layered material problem, the number of local RB atoms increased with noise level, reflecting higher posterior concentration and longer SMC path lengths.
• In the high-dimensional inclusion problem, more RB atoms were required compared to the layered case, indicating higher computational cost with increased parameter dimensionality.
• Posterior mean estimates were consistent with MCMC results, and uncertainty quantification (standard deviation) correctly increased with data noise levels (5%, 10%, 20%).
• The adaptive RB strategy maintained high accuracy with a threshold of 5% of the loss standard deviation, ensuring reliable posterior approximation without overfitting.
• The method demonstrated robustness and scalability across multiple PDE-based inverse problems, including advection-diffusion and elastography, with consistent performance relative to state-of-the-art MCMC.