[Paper Review] On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometry

Andi Han, Bamdev Mishra+2 more|arXiv (Cornell University)|Dec 6, 2021

Morphological variations and asymmetry1 citations

TL;DR

This paper compares the Bures-Wasserstein (BW) and affine-invariant (AI) geometries for Riemannian optimization on symmetric positive definite (SPD) matrices, demonstrating that BW's linear metric dependence and non-negative curvature enhance robustness and convergence, especially for ill-conditioned matrices. It further establishes that geodesic convexity is preserved under BW geometry for key cost functions.

ABSTRACT

In this paper, we comparatively analyze the Bures-Wasserstein (BW) geometry with the popular Affine-Invariant (AI) geometry for Riemannian optimization on the symmetric positive definite (SPD) matrix manifold. Our study begins with an observation that the BW metric has a linear dependence on SPD matrices in contrast to the quadratic dependence of the AI metric. We build on this to show that the BW metric is a more suitable and robust choice for several Riemannian optimization problems over ill-conditioned SPD matrices. We show that the BW geometry has a non-negative curvature, which further improves convergence rates of algorithms over the non-positively curved AI geometry. Finally, we verify that several popular cost functions, which are known to be geodesic convex under the AI geometry, are also geodesic convex under the BW geometry. Extensive experiments on various applications support our findings.

Motivation & Objective

• To evaluate the suitability of the Bures-Wasserstein (BW) geometry for Riemannian optimization on symmetric positive definite (SPD) matrices compared to the widely used affine-invariant (AI) geometry.
• To investigate how the linear metric dependence of BW contrasts with the quadratic dependence of AI, and how this affects optimization performance.
• To analyze curvature properties of the BW geometry and assess their impact on algorithm convergence rates.
• To verify whether commonly used cost functions, known to be geodesic convex under AI geometry, retain geodesic convexity under BW geometry.
• To empirically validate the theoretical advantages of BW geometry across diverse applications involving ill-conditioned SPD matrices.

Proposed method

• The paper analyzes the Bures-Wasserstein metric's structure, emphasizing its linear dependence on SPD matrices, contrasting it with the quadratic dependence of the affine-invariant metric.
• It derives and examines the curvature properties of the BW geometry, proving it has non-negative sectional curvature, unlike the AI geometry which can have negative curvature.
• Theoretical analysis is conducted to establish geodesic convexity of standard cost functions under the BW geometry, using the metric's structural properties.
• The authors compare optimization performance using both geometries on ill-conditioned SPD matrices, measuring convergence speed and robustness.
• Extensive experiments are performed on real-world applications to validate the theoretical findings, including tasks involving SPD matrix estimation and low-rank approximation.
• The study leverages Riemannian optimization algorithms such as Riemannian trust-region and conjugate gradient methods under both BW and AI geometries for comparative evaluation.

Experimental results

Research questions

• RQ1 How does the Bures-Wasserstein metric's linear dependence on SPD matrices affect optimization performance compared to the quadratic dependence of the affine-invariant metric?
• RQ2 What are the curvature properties of the Bures-Wasserstein geometry, and how do they influence convergence rates in Riemannian optimization?
• RQ3 Are standard cost functions that are geodesic convex under the affine-invariant geometry also geodesic convex under the Bures-Wasserstein geometry?
• RQ4 Does the Bures-Wasserstein geometry offer improved robustness and convergence for ill-conditioned symmetric positive definite matrices compared to the affine-invariant geometry?
• RQ5 Can the Bures-Wasserstein geometry maintain or improve performance across diverse applications involving SPD matrices?

Key findings

• The Bures-Wasserstein metric exhibits a linear dependence on SPD matrices, which simplifies optimization and enhances numerical stability compared to the quadratic dependence of the affine-invariant metric.
• The Bures-Wasserstein geometry has non-negative sectional curvature, which contributes to faster and more stable convergence of Riemannian optimization algorithms compared to the potentially negatively curved affine-invariant geometry.
• Several widely used cost functions—previously known to be geodesic convex under the affine-invariant geometry—are also geodesic convex under the Bures-Wasserstein geometry, ensuring favorable optimization behavior.
• Empirical results demonstrate that Riemannian optimization under the Bures-Wasserstein geometry achieves superior robustness and faster convergence on ill-conditioned SPD matrices compared to the affine-invariant approach.
• The study confirms that the Bures-Wasserstein geometry is a more suitable and robust choice for Riemannian optimization over the SPD manifold, particularly in challenging numerical regimes.