[Paper Review] Efficient and Guaranteed Rank Minimization by Atomic Decomposition

Kiryung Lee, Yoram Bresler|ArXiv.org|Jan 13, 2009

Sparse and Compressive Sensing Techniques12 references4 citations

TL;DR

This paper proposes ADMiRA, a novel algorithm for efficient and guaranteed low-rank matrix recovery by extending the CoSaMP framework from sparse vector recovery to low-rank matrix approximation. It uses atomic decomposition and truncated SVD to iteratively identify and refine low-rank components, achieving performance guarantees under a rank-restricted isometry property (R-RIP) and scaling efficiently to large problems via randomized low-rank approximation.

ABSTRACT

Recht, Fazel, and Parrilo provided an analogy between rank minimization and $\ell_0$-norm minimization. Subject to the rank-restricted isometry property, nuclear norm minimization is a guaranteed algorithm for rank minimization. The resulting semidefinite formulation is a convex problem but in practice the algorithms for it do not scale well to large instances. Instead, we explore missing terms in the analogy and propose a new algorithm which is computationally efficient and also has a performance guarantee. The algorithm is based on the atomic decomposition of the matrix variable and extends the idea in the CoSaMP algorithm for $\ell_0$-norm minimization. Combined with the recent fast low rank approximation of matrices based on randomization, the proposed algorithm can efficiently handle large scale rank minimization problems.

Motivation & Objective

• To address the computational inefficiency of nuclear norm minimization via semidefinite programming in large-scale rank minimization problems.
• To extend the performance guarantees of CoSaMP for $β_0$-norm minimization to the matrix rank minimization problem.
• To develop a computationally efficient algorithm that maintains theoretical recovery guarantees for low-rank matrix reconstruction.
• To enable scalable solution of large-scale rank minimization problems using randomized low-rank approximation techniques.

Proposed method

• The algorithm recasts the rank minimization problem as a rank-$ r $ matrix approximation problem, minimizing the residual $ \|\mathcal{A}X - b\|_2 $ under a rank constraint.
• It employs atomic decomposition, representing the solution as a sum of rank-one matrices, and iteratively identifies the most correlated atoms via the proxy matrix $ \mathcal{A}^* \nu $.
• At each iteration, the algorithm performs a truncated singular value decomposition (SVD) to identify the dominant singular subspace of the current proxy matrix.
• It uses a greedy selection strategy to update the candidate solution by solving a small least-squares problem over the selected subspace.
• The method incorporates a thresholding step to prune components and maintain the rank constraint, ensuring convergence to a solution with rank at most $ r $.
• It leverages randomized SVD algorithms to achieve $ O((m+n)r^3 \log r) $ per-iteration complexity, enabling scalability to large matrices.

Experimental results

Research questions

• RQ1 Can the performance guarantees of CoSaMP for sparse vector recovery be extended to low-rank matrix recovery?
• RQ2 Can a computationally efficient algorithm be designed for large-scale rank minimization that maintains theoretical recovery guarantees?
• RQ3 How does the performance of ADMiRA compare to existing methods like SVT and nuclear norm minimization in terms of accuracy and convergence speed?
• RQ4 What is the impact of the R-RIP condition on the recovery performance of ADMiRA in practical settings such as matrix completion?

Key findings

• ADMiRA achieves exact recovery of the true low-rank solution under the rank-restricted isometry property (R-RIP), with performance guarantees comparable to those of CoSaMP in the sparse vector case.
• In matrix completion experiments, ADMiRA achieved higher reconstruction SNR (82 dB) and fewer iterations (11) than SVT (79 dB, 54 iterations) for $ n=1000, r=2, p=0.2n^2 $.
• ADMiRA demonstrated superior phase transition performance compared to SVT, with higher success rates in recovering low-rank matrices across various $ (n, p, r) $ combinations.
• The algorithm scales efficiently to large problems, with per-iteration complexity $ O((m+n)r^3 \log r) $, enabled by randomized SVD and atomic decomposition.
• Despite the R-RIP not being satisfied in the matrix completion experiments, ADMiRA still achieved high-quality recovery, suggesting potential for broader applicability beyond strict R-RIP assumptions.