[Paper Review] Matrix Completion With Noise

Emmanuel J. Candès, Yaniv Plan|ArXiv.org|Mar 18, 2009

Sparse and Compressive Sensing Techniques21 references58 citations

TL;DR

This paper establishes theoretical guarantees for matrix completion under noisy conditions, showing that nuclear-norm minimization accurately recovers low-rank matrices from a nearly minimal number of noisy entries. It proves that with about $ nr\log^2 n $ noisy samples, the recovery error is proportional to the noise level, even when entries are corrupted by small noise.

ABSTRACT

On the heels of compressed sensing, a remarkable new field has very recently emerged. This field addresses a broad range of problems of significant practical interest, namely, the recovery of a data matrix from what appears to be incomplete, and perhaps even corrupted, information. In its simplest form, the problem is to recover a matrix from a small sample of its entries, and comes up in many areas of science and engineering including collaborative filtering, machine learning, control, remote sensing, and computer vision to name a few. This paper surveys the novel literature on matrix completion, which shows that under some suitable conditions, one can recover an unknown low-rank matrix from a nearly minimal set of entries by solving a simple convex optimization problem, namely, nuclear-norm minimization subject to data constraints. Further, this paper introduces novel results showing that matrix completion is provably accurate even when the few observed entries are corrupted with a small amount of noise. A typical result is that one can recover an unknown n x n matrix of low rank r from just about nr log^2 n noisy samples with an error which is proportional to the noise level. We present numerical results which complement our quantitative analysis and show that, in practice, nuclear norm minimization accurately fills in the many missing entries of large low-rank matrices from just a few noisy samples. Some analogies between matrix completion and compressed sensing are discussed throughout.

Motivation & Objective

• To establish theoretical conditions under which low-rank matrices can be accurately recovered from a small number of noisy entries.
• To extend matrix completion theory beyond the noiseless case to handle realistic scenarios with corrupted observations.
• To quantify the trade-off between sampling rate, noise level, and recovery accuracy in low-rank matrix reconstruction.
• To provide a theoretical foundation for practical applications such as collaborative filtering and system identification where data is incomplete and noisy.
• To draw parallels with compressed sensing and extend its principles to the matrix completion setting with noise.

Proposed method

• Proposes nuclear-norm minimization as the primary optimization framework for recovering low-rank matrices from incomplete and noisy data.
• Uses convex relaxation via the nuclear norm to approximate the rank minimization problem, which is NP-hard.
• Applies tools from random matrix theory and duality in optimization to derive error bounds under random sampling of entries.
• Introduces a noise-aware recovery model where observed entries are assumed to be corrupted by sub-Gaussian noise with variance $\sigma^2$.
• Derives oracle inequalities that bound the recovery error in terms of the noise level and degrees of freedom of the low-rank matrix.
• Employs a Lagrangian dual approach to analyze the recovery performance and establish high-probability error bounds.

Experimental results

Research questions

• RQ1 Can low-rank matrices be accurately recovered from a nearly minimal number of noisy entries using convex optimization?
• RQ2 What is the fundamental trade-off between the number of sampled entries, the noise level, and the reconstruction error in matrix completion?
• RQ3 How does the performance of nuclear-norm minimization compare to an ideal oracle that knows the true low-rank subspace?
• RQ4 To what extent does the theory of compressed sensing extend to matrix completion under noisy conditions?
• RQ5 What is the minimal sampling rate required to ensure stable recovery of low-rank matrices when entries are corrupted by noise?

Key findings

• Matrix completion with noise is provably accurate: an $ n \times n $ matrix of rank $ r $ can be recovered from approximately $ nr\log^2 n $ noisy samples with error proportional to the noise level.
• The recovery error is bounded by $ C \sigma \sqrt{\text{df}/m} $ with high probability, where $ \text{df} = r(2n - r) $, $ m $ is the number of observed entries, and $ \sigma $ is the noise standard deviation.
• Numerical experiments show that the actual recovery error is well-approximated by $ 1.68 \times \sqrt{\text{df}/m} $, and never exceeds $ 2.25 \times \sqrt{\text{df}/m} $.
• In a real-world test on a $ 366 \times 1472 $ temperature matrix, the algorithm achieved a relative Frobenius error of $ 16.6\% $ with 30% sampling, outperforming the best rank-2 approximation in terms of error relative to the true matrix.
• The theoretical error bounds are validated by numerical results that closely match the predicted performance, especially as the number of samples and matrix size increase.
• The proposed method achieves performance close to the oracle error (which knows the true rank-2 subspace), with a multiplicative factor of about 1.68 in practice.