[Paper Review] Improve Unscented Kalman Inversion With Low-Rank Approximation and Reduced-Order Model

Daniel Zhengyu Huang, Jiaoyang Huang|arXiv (Cornell University)|Feb 21, 2021

Seismic Imaging and Inversion Techniques69 references4 citations

TL;DR

This paper enhances the Unscented Kalman Inversion (UKI) for large-scale inverse problems by combining low-rank covariance approximation and reduced-order models (ROMs), reducing computational and memory costs while preserving accuracy. The method achieves efficiency comparable to gradient-based methods in high-dimensional problems with up to O(10^5) parameters, as demonstrated on a barotropic model and a 3D GCM.

ABSTRACT

The unscented Kalman inversion (UKI) presented in [1] is a general derivative-free approach to solving the inverse problem. UKI is particularly suitable for inverse problems where the forward model is given as a black box and may not be differentiable. The regularization strategy and convergence property of the UKI are thoroughly studied, and the method is demonstrated effectively handling noisy observation data and solving chaotic inverse problems. In this paper, we aim to make the UKI more efficient in terms of computational and memory costs for large scale inverse problems. We take advantages of the low-rank covariance structure to reduce the number of forward problem evaluations and the memory cost, related to the need to propagate large covariance matrices. And we leverage reduced-order model techniques to further speed up these forward evaluations. The effectiveness of the enhanced UKI is demonstrated on a barotropic model inverse problem with O($10^5$) unknown parameters and a 3D generalized circulation model (GCM) inverse problem, where each iteration is as efficient as that of gradient-based optimization methods.

Motivation & Objective

• Address the high computational and memory costs of UKI in large-scale inverse problems with many unknown parameters.
• Reduce the number of forward model evaluations and storage of large covariance matrices via low-rank approximation of the UKI's covariance structure.
• Accelerate forward model evaluations using reduced-order model (ROM) techniques to improve overall efficiency.
• Demonstrate that the enhanced UKI achieves computational efficiency comparable to gradient-based optimization methods in high-dimensional settings.
• Enable effective solution of large-scale, noisy, and chaotic inverse problems where forward models are expensive or black-box.

Proposed method

• Apply low-rank approximation to the UKI’s covariance matrices, reducing the number of required forward evaluations from 2Nθ+1 to a much smaller number by exploiting low-rank structure in parameter space.
• Use proper orthogonal decomposition (POD) or similar ROM techniques to construct a reduced-order forward model that approximates the full-order model with high accuracy.
• Integrate the ROM into the UKI framework to replace expensive full-order forward evaluations during each iteration, significantly reducing computational cost.
• Maintain the derivative-free nature of UKI while ensuring convergence and regularization properties through the unscented transform and stochastic dynamics.
• Leverage deterministic ensemble Kalman inversion variants (e.g., ensemble adjustment or transform Kalman inversion) to avoid explicit covariance matrix storage and improve numerical stability.
• Use matrix square root factorizations and SVD-based transformations to efficiently compute Kalman updates without storing full Nθ×Nθ covariance matrices.

Experimental results

Research questions

• RQ1 Can low-rank approximation significantly reduce the number of forward model evaluations in UKI without degrading solution accuracy?
• RQ2 To what extent can reduced-order models (ROMs) accelerate forward evaluations in UKI while preserving fidelity for large-scale inverse problems?
• RQ3 Does the combination of low-rank approximation and ROMs enable UKI to achieve computational efficiency comparable to gradient-based methods in high-dimensional settings?
• RQ4 How well does the enhanced UKI perform on chaotic or noisy inverse problems with O(10^5) unknown parameters?
• RQ5 Can the method maintain the convergence and regularization properties of UKI while drastically reducing memory and computational costs?

Key findings

• The enhanced UKI reduces computational cost and memory usage by exploiting low-rank structure in the UKI’s covariance matrices, significantly cutting the number of required forward evaluations.
• The integration of reduced-order models (ROMs) enables each UKI iteration to be as efficient as that of gradient-based optimization methods, even for problems with O(10^5) unknown parameters.
• The method successfully solves a barotropic model inverse problem with O(10^5) parameters, demonstrating scalability and robustness to noise and chaos.
• The 3D generalized circulation model (GCM) inverse problem was solved efficiently, confirming that the enhanced UKI maintains accuracy and convergence despite high dimensionality.
• The approach preserves the derivative-free, regularization, and smoothing properties of UKI, enabling effective handling of noisy and chaotic forward models.
• The method achieves linear (or superlinear) convergence in practice, consistent with the theoretical properties of UKI, even in large-scale settings.