[Paper Review] Convergence Analysis for Rectangular Matrix Completion Using Burer-Monteiro Factorization and Gradient Descent

Qinqing Zheng, John Lafferty|arXiv (Cornell University)|May 23, 2016

Sparse and Compressive Sensing Techniques28 references114 citations

TL;DR

The paper analyzes a nonconvex lifted formulation for rectangular matrix completion, proving linear convergence of gradient descent with high probability under a near-minimal number of observations.

ABSTRACT

We address the rectangular matrix completion problem by lifting the unknown matrix to a positive semidefinite matrix in higher dimension, and optimizing a nonconvex objective over the semidefinite factor using a simple gradient descent scheme. With $O( μr^2 κ^2 n \max(μ, \log n))$ random observations of a $n_1 imes n_2$ $μ$-incoherent matrix of rank $r$ and condition number $κ$, where $n = \max(n_1, n_2)$, the algorithm linearly converges to the global optimum with high probability.

Motivation & Objective

• Motivate and study rectangular matrix completion with low-rank structure using semidefinite lifting and factorization.
• Introduce a nonconvex objective over the semidefinite factor and analyze gradient descent with a spectral initializer.
• Establish conditions under which the true lifted solution is identifiable and gradient descent converges geometrically.
• Provide explicit sample complexity requirements and convergence rates in terms of rank, coherence, and condition number.

Proposed method

• Lift X* to a positive semidefinite Y* and factorize as Y*=Z Z^T with Z in R^{(n1+n2) x r}.
• Formulate a nonconvex objective f(Z) that measures lifting error plus a regularizer to align column spaces (lambda=1/2).
• Apply projected gradient descent with a closed-form incoherence projection onto set C to maintain incoherence.
• Use a spectral initialization from the top-rank factor of p^{-1}P_Omega(X*) as initialization.
• Prove linear convergence to the solution set under Bernoulli (or uniform) sampling with m >= c mu r^2 kappa^2 max(mu, log n) n observations.
• Provide a local regularity condition RC and show convergence rates depending on mu, r, kappa, and p.

Experimental results

Research questions

• RQ1 What sample complexity (in terms of mu, r, kappa, and n) suffices to identifiably recover the rectangular low-rank matrix X* under random observations?
• RQ2 Does gradient descent on the lifted nonconvex objective converge linearly to the global optimum when initialized spectrally, and under what conditions?
• RQ3 How does the regularization term and incoherence constraint influence convergence and identifiability in the lifted Burer-Monteiro factorization for rectangular matrix completion?
• RQ4 How do the proposed methods compare in theory and practice to existing convex and nonconvex approaches for matrix completion?

Key findings

• With m observations satisfying m >= c0 mu r^2 kappa^2 max(mu, log n) n, the gradient descent iterates converge geometrically to the lifted solution with high probability.
• Spectral initialization places the starting point within a small neighborhood of the solution set, enabling linear convergence under appropriate step size and regularization.
• The regularization lambda=1/2 yields a local regularity condition ensuring convergence; without it, convergence behavior may differ.
• The algorithm achieves global convergence guarantees in the Bernoulli sampling model, and the sample complexity scales with mu, r, kappa, and n but is independent of the desired accuracy in the stated results.
• Experimental results support the scalability and competitive runtime of the proposed GD approach compared to SVP, OptSpace, nuclear norm, and trust-region methods.