[Paper Review] Laplace priors and spatial inhomogeneity in Bayesian inverse problems

Sergios Agapiou, Sven Wang|arXiv (Cornell University)|Dec 10, 2021

Statistical Methods and Inference4 citations

TL;DR

This paper establishes frequentist posterior contraction rates for Laplace-prior-based Bayesian inverse problems with spatially inhomogeneous parameters, showing that Laplace priors achieve minimax-optimal rates over $B^{eta}_{11}$-smoothness classes, while Gaussian priors are limited to slower polynomially suboptimal rates. The analysis relies on local Lipschitz conditions for nonlinear PDE forward maps and novel concentration inequalities for $oldsymbol{ ext{L}}^1$-penalized wavelet estimators, which correspond to MAP estimators under Laplace priors.

ABSTRACT

Spatially inhomogeneous functions, which may be smooth in some regions and rough in other regions, are modelled naturally in a Bayesian manner using so-called Besov priors which are given by random wavelet expansions with Laplace-distributed coefficients. This paper studies theoretical guarantees for such prior measures - specifically, we examine their frequentist posterior contraction rates in the setting of non-linear inverse problems with Gaussian white noise. Our results are first derived under a general local Lipschitz assumption on the forward map. We then verify the assumption for two non-linear inverse problems arising from elliptic partial differential equations, the Darcy flow model from geophysics as well as a model for the Schrödinger equation appearing in tomography. In the course of the proofs, we also obtain novel concentration inequalities for penalized least squares estimators with $\ell^1$ wavelet penalty, which have a natural interpretation as maximum a posteriori (MAP) estimators. The true parameter is assumed to belong to some spatially inhomogeneous Besov class $B^α_{11}$, $α>0$. In a setting with direct observations, we complement these upper bounds with a lower bound on the rate of contraction for arbitrary Gaussian priors. An immediate consequence of our results is that while Laplace priors can achieve minimax-optimal rates over $B^α_{11}$-classes, Gaussian priors are limited to a (by a polynomial factor) slower contraction rate. This gives information-theoretical justification for the intuition that Laplace priors are more compatible with $\ell^1$ regularity structure in the underlying parameter.

Motivation & Objective

• To establish theoretical posterior contraction rates for Bayesian inverse problems with spatially inhomogeneous parameters modeled via Besov priors.
• To analyze the performance of Laplace-distributed wavelet coefficient priors in non-linear inverse problems with Gaussian white noise.
• To compare the frequentist contraction rates of Laplace priors versus Gaussian priors in recovering parameters from $B^{eta}_{11}$-smoothness classes.
• To verify the local Lipschitz condition for forward maps arising from elliptic PDEs, including the Darcy flow and Schrödinger equations.
• To derive novel concentration inequalities for $\ell^1$-penalized least squares estimators, linking them to MAP estimation under Laplace priors.

Proposed method

• The study employs Besov prior measures defined by random wavelet expansions with i.i.d. Laplace-distributed coefficients to model spatially inhomogeneous functions in $B^{eta}_{11}$-classes.
• A general framework is developed for posterior contraction rates under a local Lipschitz condition on the forward map $\mathcal{G}$, applicable to non-linear PDE-based inverse problems.
• Novel concentration inequalities are derived for $\ell^1$-penalized least squares estimators, which are shown to correspond to MAP estimators under Laplace wavelet priors.
• The local Lipschitz condition is verified for two non-linear PDE models: the steady-state Darcy flow and the Schrödinger equation in tomography.
• Theoretical analysis uses wavelet characterizations of Besov spaces and embedding inequalities between $H^\alpha$ and $B^\beta_{11}$ norms.
• A lower bound on contraction rates is derived for arbitrary Gaussian priors under direct observation, demonstrating their inherent limitation compared to Laplace priors.

Experimental results

Research questions

• RQ1 Can Laplace priors achieve minimax-optimal posterior contraction rates in non-linear Bayesian inverse problems with spatially inhomogeneous parameters?
• RQ2 What is the theoretical performance gap between Laplace priors and Gaussian priors in terms of posterior contraction rates for $B^\beta_{11}$-smoothness classes?
• RQ3 Under what conditions on the forward map $\mathcal{G}$ does the posterior contract at optimal rates when using Laplace wavelet priors?
• RQ4 How do $\ell^1$-penalized least squares estimators, interpreted as MAP estimators, concentrate around the true parameter in high-dimensional wavelet spaces?
• RQ5 To what extent do Gaussian priors fail to achieve optimal rates in the presence of spatial inhomogeneity, and why?

Key findings

• Laplace priors achieve minimax-optimal posterior contraction rates over $B^{\beta}_{11}$-smoothness classes when $\beta > 0$ is sufficiently large.
• Gaussian priors are limited to a (by a polynomial factor) slower contraction rate than the minimax-optimal rate, even under direct observation.
• The local Lipschitz condition on the forward map $\mathcal{G}$ is verified for both the Darcy flow and Schrödinger equation models, enabling the application of the general contraction rate theory.
• Novel concentration inequalities for $\ell^1$-penalized least squares estimators are established, providing a theoretical foundation for the MAP estimator under Laplace priors.
• The results provide information-theoretic justification that Laplace priors are more compatible with $\ell^1$-regularity structures in the parameter space than Gaussian priors.
• The analysis reveals that the choice of prior is critical in capturing spatial inhomogeneity, with Laplace priors uniquely suited to model functions with mixed smoothness and discontinuities.