[Paper Review] Matrix Completion from a Few Entries

Raghunandan H. Keshavan, Andrea Montanari+1 more|ArXiv.org|Jan 20, 2009

Random Matrices and Applications16 references98 citations

TL;DR

This paper proposes an efficient matrix completion algorithm that reconstructs low-rank matrices from a small number of uniformly random entries by combining trimming of high-degree rows/columns with spectral projection and residual cleaning. It proves that exact recovery is possible with O(n log n) entries when rank r = O(1), and provides an RMSE bound of C(α)(nr/|E|)^{1/2} for general rank r, significantly improving prior guarantees under incoherence assumptions.

ABSTRACT

Let M be a random (alpha n) x n matrix of rank r<

Motivation & Objective

• To address the fundamental problem of reconstructing a low-rank matrix from a small, uniformly random subset of its entries.
• To develop an efficient algorithm that outperforms naive spectral projection methods, especially in the presence of high-degree rows/columns that distort singular vectors.
• To establish theoretical bounds on the number of required entries for exact or accurate matrix recovery under incoherence conditions.
• To generalize spectral results on sparse random matrices to support the analysis of the proposed algorithm.
• To provide a complexity-efficient method with O(|E|r log n) runtime, suitable for massive datasets.

Proposed method

• Trims the observed matrix by zeroing out rows and columns with degree exceeding 2|E|/m and 2|E|/n, respectively, to eliminate artifacts from high-degree entries.
• Applies spectral projection by computing the SVD of the trimmed matrix and retaining only the top r singular values and vectors, scaled by (mn)/|E| to correct for sparsity.
• Performs residual cleaning via optimization over low-rank factors X and Y to minimize the discrepancy F(X,Y) between the reconstructed matrix and observed entries.
• Employs incoherence conditions on the factor matrices U and V to ensure uniform distribution of information across rows and columns.
• Uses concentration inequalities and spectral gap analysis to bound the deviation of singular vectors from the true underlying structure.
• Introduces a novel generalization of Friedman-Kahn-Szemerédi and Feige-Ofek results on sparse random matrix spectra to analyze the trimmed matrix's spectral properties.

Experimental results

Research questions

• RQ1 Can a low-rank matrix be accurately reconstructed from a small number of uniformly random entries, and what is the minimal number of entries required for exact recovery?
• RQ2 Why does standard spectral projection fail when the observed entries are unevenly distributed across rows and columns?
• RQ3 Can trimming high-degree rows and columns significantly improve the performance of matrix completion algorithms?
• RQ4 What theoretical guarantees can be established for the reconstruction error in terms of the number of observed entries and matrix dimensions?
• RQ5 How does the algorithm's complexity scale with the number of observed entries and the rank of the matrix?

Key findings

• The algorithm achieves an RMSE bound of C(α)(nr/|E|)^{1/2}, which improves upon prior guarantees for matrix completion under incoherence.
• Exact matrix recovery is possible with high probability using only O(n log n) observed entries when the rank r is bounded.
• Trimming high-degree rows and columns significantly enhances spectral structure recovery, making the underlying low-rank structure more apparent.
• The algorithm runs in O(|E|r log n) time, making it scalable to massive datasets.
• Theoretical analysis relies on a generalized spectral gap result for sparse random matrices, extending prior work by Friedman-Kahn-Szemerédi and Feige-Ofek.
• The method is robust under incoherence assumptions, which hold with high probability for random or i.i.d. factor matrices.