[Paper Review] Diagonal and Low-Rank Matrix Decompositions, Correlation Matrices, and Ellipsoid Fitting

James Saunderson, Venkat Chandrasekaran+2 more|Apr 5, 2012

Sparse and Compressive Sensing TechniquesEngineering1 references3 citations

TL;DR

This paper establishes an equivalence between three problems: decomposing a matrix into a diagonal and low-rank component, analyzing the facial structure of correlation matrices (the elliptope), and determining whether a centered ellipsoid passes through a given set of points. It proves that minimum trace factor analysis can exactly recover low-rank positive semidefinite matrices from their sum with any diagonal matrix when the subspace coherence is below 1/2, providing a provable recovery condition for high-dimensional matrix decompositions.

ABSTRACT

In this paper we establish links between, and new results for, three problems that are not usually considered together. The first is a matrix decomposition problem that arises in areas such as statistical modeling and signal processing: given a matrix $X$ formed as the sum of an unknown diagonal matrix and an unknown low rank positive semidefinite matrix, decompose $X$ into these constituents. The second problem we consider is to determine the facial structure of the set of correlation matrices, a convex set also known as the elliptope. This convex body, and particularly its facial structure, plays a role in applications from combinatorial optimization to mathematical finance. The third problem is a basic geometric question: given points $v_1,v_2,...,v_n\in \R^k$ (where $n > k$) determine whether there is a centered ellipsoid passing \emph{exactly} through all of the points. We show that in a precise sense these three problems are equivalent. Furthermore we establish a simple sufficient condition on a subspace $U$ that ensures any positive semidefinite matrix $L$ with column space $U$ can be recovered from $D+L$ for any diagonal matrix $D$ using a convex optimization-based heuristic known as minimum trace factor analysis. This result leads to a new understanding of the structure of rank-deficient correlation matrices and a simple condition on a set of points that ensures there is a centered ellipsoid passing through them.

Motivation & Objective

• To establish a theoretical equivalence between matrix decomposition into diagonal and low-rank components, facial structure analysis of the elliptope (set of correlation matrices), and ellipsoid fitting through given points.
• To provide a provable recovery condition for the minimum trace factor analysis (MTFA) heuristic in matrix decomposition.
• To characterize when a centered ellipsoid can pass exactly through a given set of points in R^k (n > k).
• To derive a sufficient condition based on subspace coherence for exact recovery of low-rank matrices via convex optimization.
• To connect the facial structure of the elliptope to the geometric problem of ellipsoid fitting and matrix decomposition.

Proposed method

• Formulates the matrix decomposition problem as finding a low-rank positive semidefinite matrix L and diagonal matrix D such that X = D + L.
• Uses semidefinite programming duality to link the MTFA heuristic to the facial structure of the elliptope.
• Introduces the concept of subspace coherence μ(U) as a key geometric quantity to determine recovery feasibility.
• Applies Walters' theorem on linear systems with non-negative matrices to prove existence of non-negative solutions to a system derived from the dual problem.
• Establishes that if μ(U) < 1/2, then MTFA recovers L exactly from X = D + L for any diagonal D.
• Employs beta distribution tail bounds to show that random subspaces with dimension r = (1/2 - ε)n have μ(U) < 1/2 with high probability when r > 3/ε².

Experimental results

Research questions

• RQ1 Under what conditions can a matrix X = D + L be exactly decomposed into a diagonal matrix D and a low-rank positive semidefinite matrix L using convex optimization?
• RQ2 What is the relationship between the facial structure of the elliptope and the existence of a centered ellipsoid passing through a given set of points?
• RQ3 When is the minimum trace factor analysis (MTFA) heuristic guaranteed to recover the low-rank component L from X = D + L?
• RQ4 How does the coherence of the column space of L affect the recoverability of L via MTFA?
• RQ5 What geometric condition on a set of points in R^k ensures the existence of a centered ellipsoid passing through all of them?

Key findings

• If the subspace coherence μ(U) of the column space of the low-rank matrix L is less than 1/2, then minimum trace factor analysis exactly recovers L from X = D + L for any diagonal matrix D.
• The problem of ellipsoid fitting through n points in R^k (n > k) is equivalent to the matrix decomposition and elliptope facial structure problems.
• A necessary and sufficient condition for the existence of a non-negative solution to the linear system derived from the dual of the MTFA problem is μ(U) < 1/2.
• For a random r-dimensional subspace of R^n with r = (1/2 - ε)n and r > 3/ε², the probability that μ(U) ≥ 1/2 decays exponentially with n.
• The facial structure of the elliptope is directly linked to the geometric realizability of ellipsoids through given points.
• The paper provides a new, simple sufficient condition for exact recovery in high-dimensional factor analysis, improving on prior heuristic or algebraic approaches.