[Paper Review] Tight oracle bounds for low-rank matrix recovery from a minimal number of random measurements

Emmanuel J. Candès, Yaniv Plan|arXiv (Cornell University)|Jan 2, 2010

Sparse and Compressive Sensing Techniques25 references141 citations

TL;DR

This paper establishes tight oracle bounds for low-rank matrix recovery using nuclear-norm minimization from a near-minimal number of random linear measurements. It proves that recovery error is within a constant factor of the minimax risk and idealized oracle error, even with noisy data, and extends to full-rank matrices with decaying singular values.

ABSTRACT

This paper presents several novel theoretical results regarding the recovery of a low-rank matrix from just a few measurements consisting of linear combinations of the matrix entries. We show that properly constrained nuclear-norm minimization stably recovers a low-rank matrix from a constant number of noisy measurements per degree of freedom; this seems to be the first result of this nature. Further, the recovery error from noisy data is within a constant of three targets: 1) the minimax risk, 2) an oracle error that would be available if the column space of the matrix were known, and 3) a more adaptive oracle error which would be available with the knowledge of the column space corresponding to the part of the matrix that stands above the noise. Lastly, the error bounds regarding low-rank matrices are extended to provide an error bound when the matrix has full rank with decaying singular values. The analysis in this paper is based on the restricted isometry property (RIP) introduced in [6] for vectors, and in [22] for matrices.

Motivation & Objective

• To establish theoretical guarantees for stable recovery of low-rank matrices from a minimal number of random linear measurements.
• To show that nuclear-norm minimization achieves error bounds within a constant factor of the minimax risk and oracle error, rather than a logarithmic factor as in prior work.
• To extend the analysis to full-rank matrices with decaying singular values, capturing well-approximated low-rank structures.
• To demonstrate that the number of measurements required is within a constant factor of the information-theoretic lower bound, without extra logarithmic factors.

Proposed method

• Uses the restricted isometry property (RIP) for matrices, extending the vector case to low-rank matrix recovery.
• Applies nuclear-norm minimization as a convex relaxation to recover low-rank matrices from linear measurements.
• Employs RIP-based analysis to derive error bounds that are tight relative to the minimax risk and oracle error.
• Derives bounds on the eigenvalues of the measurement operator restricted to low-rank subspaces, ensuring stability.
• Uses a rescaling argument and SVD decomposition to reduce the problem to estimating a coefficient vector under noise.
• Applies Lemma 3.11 and Lemma 3.12 to lower-bound the minimax risk and relate it to the measurement operator’s spectral properties.

Experimental results

Research questions

• RQ1 Can low-rank matrix recovery be stably achieved from a number of measurements close to the information-theoretic minimum?
• RQ2 Is the recovery error via nuclear-norm minimization within a constant factor of the minimax risk and oracle error, rather than a logarithmic factor?
• RQ3 Can the error bounds be extended to full-rank matrices with decaying singular values?
• RQ4 Does the number of required measurements scale linearly with the degrees of freedom, without logarithmic penalties?

Key findings

• Nuclear-norm minimization stably recovers low-rank matrices from a constant number of noisy measurements per degree of freedom.
• The recovery error is within a constant factor of the minimax risk, which is the best possible error achievable without prior knowledge of the column space.
• The error is also within a constant factor of an 'oracle' error that assumes knowledge of the true column space of the matrix.
• The number of measurements required is within a constant factor of the theoretical lower bound of $(n_1 + n_2 - r)r$, with no logarithmic overhead.
• The results extend to full-rank matrices that are well-approximated by low-rank matrices, with error bounds depending on the decay of singular values.
• The analysis confirms that random linear measurements (not just entrywise sampling) can achieve near-optimal recovery with minimal sample complexity.