[Paper Review] Probabilistic Numerical Methods for Partial Differential Equations and Bayesian Inverse Problems

Jon Cockayne, Chris J. Oates+2 more|arXiv (Cornell University)|May 25, 2016

Gaussian Processes and Bayesian Inference4 citations

TL;DR

This paper introduces a probabilistic numerical method for solving partial differential equations (PDEs) by treating discretisation error as epistemic uncertainty, represented through a probability distribution over solutions. It enables robust Bayesian inverse problems by propagating this uncertainty through statistical inference, ensuring valid conclusions even with coarse discretisations, and frames PDE solver selection as a Bayesian experimental design problem to minimise inference error.

ABSTRACT

This paper develops a probabilistic numerical method for solution of partial differential equations (PDEs) and studies application of that method to PDE-constrained inverse problems. This approach enables the solution of challenging inverse problems whilst accounting, in a statistically principled way, for the impact of discretisation error due to numerical solution of the PDE. In particular, the approach confers robustness to failure of the numerical PDE solver, with statistical inferences driven to be more conservative in the presence of substantial discretisation error. Going further, the problem of choosing a PDE solver is cast as a problem in the Bayesian design of experiments, where the aim is to minimise the impact of solver error on statistical inferences; here the challenge of non-linear PDEs is also considered. The method is applied to parameter inference problems in which discretisation error in non-negligible and must be accounted for in order to reach conclusions that are statistically valid.

Motivation & Objective

• Address the challenge of discretisation error in PDE-constrained inverse problems, where classical methods assume perfect solution of the PDE.
• Develop a statistical framework that treats numerical solution error as epistemic uncertainty, enabling uncertainty quantification in downstream inference.
• Ensure statistical validity of inverse problem results by accounting for solver error in parameter estimation, especially when discretisation error is non-negligible.
• Reframe the choice of PDE solver as a Bayesian design of experiments problem to optimise inference robustness under uncertainty.
• Provide theoretical guarantees on the convergence of posterior distributions in inverse problems when using the probabilistic PDE solver.

Proposed method

• Construct a probabilistic numerical method (PNM) that outputs a probability distribution over PDE solutions, encoding epistemic uncertainty from discretisation.
• Model the PDE solution as a random variable with a prior distribution over a reproducing kernel Hilbert space (RKHS), incorporating smoothness and regularity assumptions.
• Use a finite-dimensional approximation of the PDE solution via a Galerkin method with a randomised basis, leading to a posterior distribution over the solution coefficients.
• Propagate the resulting uncertainty through the likelihood function in Bayesian inverse problems, ensuring coherent uncertainty propagation to parameter posteriors.
• Apply concentration inequalities and matrix norm bounds to derive error bounds on the difference between the true and approximate log-posterior in inverse problems.
• Frame PDE solver selection as a Bayesian experimental design problem, minimising the impact of solver error on statistical inference via expected information gain.

Experimental results

Research questions

• RQ1 Can a probabilistic numerical method for PDEs provide a statistically principled way to quantify and propagate discretisation error in inverse problems?
• RQ2 How does the use of a PNM affect the robustness and validity of Bayesian inference when the PDE is solved with coarse discretisations?
• RQ3 To what extent can the choice of PDE solver be optimised as a Bayesian experimental design problem to reduce uncertainty in inverse problem outcomes?
• RQ4 What theoretical guarantees can be established on the convergence of the posterior distribution in inverse problems when using a PNM with bounded discretisation error?
• RQ5 How does the proposed method handle non-linear PDEs, and what are the implications for uncertainty quantification in complex models?

Key findings

• The probabilistic PDE solver produces a posterior distribution over solutions that quantifies epistemic uncertainty due to discretisation, enabling uncertainty propagation to inverse problem inferences.
• The method ensures statistical validity of inverse problem results by accounting for solver error, leading to more conservative inferences when discretisation error is high.
• Theoretical analysis shows that the Hellinger distance between the true and approximate posterior distributions in inverse problems decays at a rate of $ O(h^{eta - ho - d/2}) $, where $ h $ is the mesh size.
• The error bound on the log-posterior difference is derived using matrix norm inequalities and Sobolev-type regularity assumptions on the solution, with explicit dependence on the data and prior regularity.
• The framework allows for the selection of a PDE solver via Bayesian experimental design, minimising the impact of discretisation error on statistical conclusions.
• The method is applicable to non-linear PDEs and maintains theoretical consistency under standard regularity and smoothness conditions on the solution and data.