[Paper Review] A Nonconvex Free Lunch for Low-Rank plus Sparse Matrix Recovery

Xiao Zhang, Lingxiao Wang+1 more|arXiv (Cornell University)|Feb 21, 2017

Sparse and Compressive Sensing Techniques46 references3 citations

TL;DR

This paper proposes a nonconvex optimization framework for low-rank plus sparse matrix recovery using matrix factorization and projected gradient descent with a double thresholding operator. It achieves locally linear convergence and matches the best-known robustness to sparsity, under a novel structural Lipschitz gradient condition that enables fast convergence for superposition-structured models.

ABSTRACT

We propose a unified framework to solve general low-rank plus sparse matrix recovery problems based on matrix factorization, which covers a broad family of objective functions satisfying the restricted strong convexity and smoothness conditions. Based on projected gradient descent and the double thresholding operator, our proposed generic algorithm is guaranteed to converge to the unknown low-rank and sparse matrices at a locally linear rate, while matching the best-known robustness guarantee (i.e., tolerance for sparsity). At the core of our theory is a novel structural Lipschitz gradient condition for low-rank plus sparse matrices, which is essential for proving the linear convergence rate of our algorithm, and we believe is of independent interest to prove fast rates for general superposition-structured models. We illustrate the application of our framework through two concrete examples: robust matrix sensing and robust PCA. Experiments on both synthetic and real datasets corroborate our theory.

Motivation & Objective

• To address the challenge of recovering low-rank and sparse matrix components from noisy or incomplete observations in a unified, nonconvex optimization framework.
• To establish a convergence guarantee with a locally linear rate for general low-rank plus sparse matrix recovery problems.
• To achieve the best-known robustness to sparsity, matching the theoretical tolerance limits of existing methods.
• To introduce a novel structural Lipschitz gradient condition tailored for superposition-structured matrix models.
• To demonstrate the framework’s effectiveness on real and synthetic data through robust matrix sensing and robust PCA applications.

Proposed method

• The framework employs matrix factorization to parameterize the low-rank component, enabling nonconvex optimization over low-dimensional manifolds.
• Projected gradient descent is used to optimize the objective function, with projections ensuring iterates remain within the low-rank manifold.
• A double thresholding operator is introduced to simultaneously promote low-rank and sparse structures in the solution.
• The convergence analysis relies on a novel structural Lipschitz gradient condition that captures the geometry of low-rank plus sparse matrices.
• The method is applicable to a broad class of objective functions satisfying restricted strong convexity and smoothness.
• Theoretical guarantees are derived under standard assumptions on the measurement operator and noise model.

Experimental results

Research questions

• RQ1 Can a nonconvex optimization framework achieve locally linear convergence for low-rank plus sparse matrix recovery?
• RQ2 What structural conditions on the objective function enable fast convergence rates in superposition-structured models?
• RQ3 How does the proposed method compare to existing convex or nonconvex approaches in terms of robustness to sparsity?
• RQ4 Can the framework be applied to concrete problems like robust matrix sensing and robust PCA with theoretical guarantees?
• RQ5 What is the role of the novel structural Lipschitz gradient condition in enabling fast convergence?

Key findings

• The proposed algorithm converges to the true low-rank and sparse components at a locally linear rate under standard assumptions.
• The method achieves the best-known robustness to sparsity, matching the theoretical tolerance limits of prior work.
• The novel structural Lipschitz gradient condition is essential for proving the linear convergence rate and is of independent interest for general superposition-structured models.
• Empirical results on synthetic and real datasets confirm the theoretical convergence and robustness claims.
• The framework successfully generalizes to robust matrix sensing and robust PCA, demonstrating practical utility.
• The double thresholding operator effectively balances low-rank and sparse recovery without requiring prior knowledge of sparsity levels.