[Paper Review] Positive definite matrices and the Symmetric Stein Divergence

Suvrit Sra|arXiv (Cornell University)|Oct 8, 2011

Mathematical Inequalities and Applications45 citations

TL;DR

This paper introduces the S-Divergence, a novel distance-like function on the cone of positive definite matrices that is the square of a metric and inherits key geometric properties of the Riemannian distance, while being computationally more efficient. Despite its nonconvexity, the S-Divergence enables global optimization for computing matrix means and medians, supported by theoretical analysis and numerical experiments.

ABSTRACT

Positive definite matrices abound in a dazzling variety of applications. This ubiquity can be in part attributed to their rich geometric structure: positive definite matrices form a self-dual convex cone whose strict interior is a Riemannian manifold. The manifold view is endowed with a natural distance function while the conic view is not. Nevertheless, drawing motivation from the conic view, we introduce the S-Divergence as a natural distance-like function on the open cone of positive definite matrices. We motivate the S-divergence via a sequence of results that connect it to the Riemannian distance. In particular, we show that (a) this divergence is the square of a distance; and (b) that it has several geometric properties similar to those of the Riemannian distance, though without being computationally as demanding. The S-divergence is even more intriguing: although nonconvex, we can still compute matrix means and medians using it to global optimality. We complement our results with some numerical experiments illustrating our theorems and our optimization algorithm for computing matrix medians.

Motivation & Objective

• To develop a computationally efficient alternative to the Riemannian distance on the manifold of positive definite matrices.
• To bridge the geometric insights from the conic structure of positive definite matrices with practical distance-like functions.
• To enable global optimization of matrix means and medians using a nonconvex divergence by leveraging its favorable geometric and algebraic properties.
• To establish theoretical connections between the S-Divergence and the Riemannian distance, showing it behaves like a squared distance with similar geometric behavior.

Proposed method

• Define the S-Divergence as a divergence function derived from the conic structure of positive definite matrices, motivated by the geometry of the self-dual convex cone.
• Prove that the S-Divergence is the square of a metric, establishing its foundation as a distance-like function.
• Demonstrate that the S-Divergence shares key geometric properties with the Riemannian distance, such as symmetry and invariance under certain matrix transformations.
• Develop an optimization algorithm for computing matrix means and medians using the S-Divergence, leveraging its structure to achieve global optimality despite nonconvexity.
• Use numerical experiments to validate the theoretical results and illustrate the efficiency and accuracy of the proposed optimization method.

Experimental results

Research questions

• RQ1 Can a divergence be constructed from the conic structure of positive definite matrices that behaves like a squared Riemannian distance but is computationally cheaper?
• RQ2 Does the S-Divergence preserve essential geometric properties of the Riemannian distance, such as symmetry and invariance?
• RQ3 Is it possible to compute matrix means and medians using a nonconvex divergence like the S-Divergence while guaranteeing global optimality?
• RQ4 How does the S-Divergence compare to the Riemannian distance in terms of computational cost and geometric fidelity?

Key findings

• The S-Divergence is proven to be the square of a metric, establishing it as a valid distance-like function on the open cone of positive definite matrices.
• The S-Divergence inherits several geometric properties of the Riemannian distance, such as symmetry and invariance under congruence transformations, despite being computationally less demanding.
• Despite the S-Divergence being nonconvex, the paper demonstrates that matrix means and medians can still be computed to global optimality using this divergence.
• Numerical experiments confirm the theoretical claims, showing that the proposed optimization algorithm converges efficiently and accurately to optimal solutions.