[Paper Review] Multifidelity Covariance Estimation via Regression on the Manifold of Symmetric Positive Definite Matrices

Aimee Maurais, Terrence Alsup+2 more|arXiv (Cornell University)|Jul 23, 2023

Morphological variations and asymmetry5 citations

TL;DR

This paper proposes a multifidelity covariance estimator that formulates covariance estimation as a regression problem on the manifold of symmetric positive definite (SPD) matrices, ensuring positive definiteness by construction. By leveraging Riemannian geometry and minimizing a Mahalanobis distance on the tangent space, the method achieves up to a 53% reduction in mean relative error compared to high-fidelity-only estimation, outperforming existing multifidelity and single-fidelity estimators in numerical experiments.

ABSTRACT

We introduce a multifidelity estimator of covariance matrices formulated as the solution to a regression problem on the manifold of symmetric positive definite matrices. The estimator is positive definite by construction, and the Mahalanobis distance minimized to obtain it possesses properties enabling practical computation. We show that our manifold regression multifidelity (MRMF) covariance estimator is a maximum likelihood estimator under a certain error model on manifold tangent space. More broadly, we show that our Riemannian regression framework encompasses existing multifidelity covariance estimators constructed from control variates. We demonstrate via numerical examples that the MRMF estimator can provide significant decreases, up to one order of magnitude, in squared estimation error relative to both single-fidelity and other multifidelity covariance estimators. Furthermore, preservation of positive definiteness ensures that our estimator is compatible with downstream tasks, such as data assimilation and metric learning, in which this property is essential.

Motivation & Objective

• Address the challenge of small-sample covariance estimation in high-dimensional, computationally expensive settings where high-fidelity data are scarce.
• Overcome the limitation of standard multifidelity methods that may produce non-positive definite covariance matrices by embedding estimation in the Riemannian geometry of symmetric positive definite matrices.
• Develop a framework that unifies existing control variate-based multifidelity estimators within a principled Riemannian regression formulation.
• Ensure compatibility with downstream applications—such as data assimilation and metric learning—that require positive definite covariance matrices.
• Demonstrate significant improvements in estimation accuracy, particularly in terms of mean relative error (MRE), across multiple numerical benchmarks.

Proposed method

• Formulate the multifidelity covariance estimation problem as a regression on the manifold of symmetric positive definite (SPD) matrices using affine-invariant Riemannian geometry.
• Define the Mahalanobis distance on the tangent space of the SPD manifold to measure estimation error, enabling practical computation via Riemannian optimization.
• Construct the estimator as the solution to a minimization problem that balances fidelity to high-fidelity samples and variance reduction via low-fidelity samples.
• Ensure the resulting estimator is positive definite by design, leveraging the intrinsic geometry of the SPD manifold rather than post-hoc correction.
• Establish theoretical connections by showing the estimator is equivalent to a maximum likelihood estimator under a specific error model on the tangent space.
• Generalize the framework to other Riemannian manifolds by adapting mean and covariance definitions from [44] to different geometric structures.

Experimental results

Research questions

• RQ1 Can a multifidelity covariance estimator be constructed that maintains positive definiteness while reducing estimation error in small-sample regimes?
• RQ2 How does the proposed Riemannian regression framework on the SPD manifold compare to existing multifidelity estimators in terms of mean relative error and variance reduction?
• RQ3 To what extent does the manifold-based Mahalanobis distance enable stable and efficient computation of the estimator without relying on vector space assumptions?
• RQ4 Can the proposed framework be generalized to other Riemannian manifolds beyond SPD matrices, such as rotation matrices or probability measures?
• RQ5 What is the theoretical justification for the estimator’s performance, particularly in relation to maximum likelihood estimation on the tangent space?

Key findings

• The manifold regression multifidelity (MRMF) estimator achieves a 53% reduction in mean relative error (MRE) compared to high-fidelity-only estimation, demonstrating significant improvement in accuracy.
• The MRMF estimator reduces MRE by 25% compared to the low-fidelity-only estimator and by 6.9% compared to the LEMF estimator, outperforming both in all tested scenarios.
• The estimator maintains positive definiteness by construction, ensuring compatibility with downstream applications such as data assimilation and metric learning.
• Theoretical analysis shows the MRMF estimator is equivalent to a maximum likelihood estimator under a specific error model on the tangent space of the SPD manifold.
• The framework generalizes existing control variate-based multifidelity estimators and provides a unified approach to multifidelity estimation on Riemannian manifolds.
• Numerical experiments confirm that the MRMF estimator reduces squared estimation error by up to one order of magnitude compared to single-fidelity and other multifidelity methods.