[Paper Review] A note on the Karhunen-Loève expansions for infinite-dimensional Bayesian inverse problems

Jinglai Li|arXiv (Cornell University)|Dec 30, 2014

Markov Chains and Monte Carlo Methods10 references3 citations

TL;DR

This paper establishes error bounds for truncated Karhunen-Loève (K-L) expansions in infinite-dimensional Bayesian inverse problems, showing that the approximation error between the true solution and its finite-dimensional K-L projection is bounded by the square root of the largest neglected eigenvalue of the prior covariance operator. The analysis relies on local Lipschitz continuity of the data misfit functional and provides a priori error estimates without requiring knowledge of the true solution.

ABSTRACT

In this note, we consider the truncated Karhunen-Loève expansion for approximating solutions to infinite dimensional inverse problems. We show that, under certain conditions, the bound of the error between a solution and its finite-dimensional approximation can be estimated without the knowledge of the solution.

Motivation & Objective

• To rigorously justify the use of truncated Karhunen-Loève expansions for approximating solutions to infinite-dimensional Bayesian inverse problems.
• To address the lack of theoretical justification for finite-dimensional approximations in practical implementations of K-L expansions.
• To derive error bounds between the true solution and its finite-dimensional K-L approximation without requiring knowledge of the true solution.
• To establish convergence rates for the K-L approximation in terms of the eigenvalues of the prior covariance operator.

Proposed method

• Formulates the Bayesian inverse problem as a minimization of the Onsager-Machlup functional in the Cameron-Martin space associated with a Gaussian prior.
• Applies a change of variables $ u = Q^{1/2}x $ to transform the infinite-dimensional optimization into a problem over the Hilbert space $ X $, enabling the use of K-L expansions.
• Uses the eigenfunction expansion of the prior covariance operator $ Q $ to project the solution onto a finite-dimensional subspace spanned by the first $ n $ eigenfunctions.
• Establishes error bounds via local Lipschitz continuity of the data misfit functional $ ilde{\Phi}(u) $, relating the error in the $ x $-space to the tail eigenvalues $ \lambda_n^* = \max_{k>n} \lambda_k $.
• Derives the key inequality $ \|x - x_n\|_X \leq L \sqrt{\lambda_n^*} $, where $ L $ depends on the local Lipschitz constant of $ \Phi $, and translates this to the original solution space via $ u = Q^{1/2}x $.
• Provides bounds on the optimality gap for the finite-dimensional problem, showing $ J(x_n') \leq \min J(x) + L^2 \lambda_n^* $, ensuring convergence of the finite-dimensional solution to the true solution.

Experimental results

Research questions

• RQ1 Can the error between the true solution of an infinite-dimensional Bayesian inverse problem and its truncated K-L approximation be bounded without knowledge of the true solution?
• RQ2 How does the convergence rate of the K-L approximation depend on the eigenvalues of the prior covariance operator?
• RQ3 Is the finite-dimensional solution obtained via K-L truncation a good approximation to the true solution in terms of the objective function value?
• RQ4 Under what conditions can the K-L expansion be rigorously justified as a dimension-reduction technique in infinite-dimensional inverse problems?

Key findings

• The error in the $ x $-space between the true minimizer $ x $ and its truncated K-L approximation $ x_n $ is bounded by $ \|x - x_n\|_X \leq L \sqrt{\lambda_n^*} $, where $ \lambda_n^* = \max_{k>n} \lambda_k $ and $ L $ is a constant depending on the local Lipschitz constant of the data misfit functional.
• The corresponding error in the original solution space $ u = Q^{1/2}x $ satisfies $ \|u - u_n\|_X < L \lambda_n^* $, showing a linear dependence on the largest neglected eigenvalue.
• The finite-dimensional solution $ x_n' $ to the truncated problem satisfies $ J(x_n') \leq \min J(x) + L^2 \lambda_n^* $, indicating that the objective function value converges to the true minimum at a rate proportional to $ \lambda_n^* $.
• The error bounds are derived under the assumption that the data misfit functional $ \Phi(u) $ is locally Lipschitz continuous, which holds for typical inverse problems such as those with Gaussian noise.
• The results provide a priori error estimates that do not require knowledge of the true solution, making them useful for adaptive dimension selection in numerical implementations.
• The theoretical framework justifies the widespread use of K-L expansions in practice by quantifying their approximation error in terms of the spectral decay of the prior covariance operator.