[Paper Review] Restricted strong convexity and weighted matrix completion: Optimal bounds with noise

Sahand Negahban, Martin J. Wainwright|arXiv (Cornell University)|Sep 10, 2010

Sparse and Compressive Sensing TechniquesEngineering36 references434 citations

TL;DR

This paper establishes restricted strong convexity (RSC) for weighted matrix completion under noisy sampling, proving non-asymptotic error bounds in the weighted Frobenius norm. It introduces an M-estimator combining data fidelity and a weighted nuclear norm, achieving optimal recovery rates under relaxed spikiness and low-rankness conditions compared to prior work.

ABSTRACT

We consider the matrix completion problem under a form of row/column weighted entrywise sampling, including the case of uniform entrywise sampling as a special case. We analyze the associated random observation operator, and prove that with high probability, it satisfies a form of restricted strong convexity with respect to weighted Frobenius norm. Using this property, we obtain as corollaries a number of error bounds on matrix completion in the weighted Frobenius norm under noisy sampling and for both exact and near low-rank matrices. Our results are based on measures of the "spikiness" and "low-rankness" of matrices that are less restrictive than the incoherence conditions imposed in previous work. Our technique involves an $M$-estimator that includes controls on both the rank and spikiness of the solution, and we establish non-asymptotic error bounds in weighted Frobenius norm for recovering matrices lying with $\ell_q$-"balls" of bounded spikiness. Using information-theoretic methods, we show that no algorithm can achieve better estimates (up to a logarithmic factor) over these same sets, showing that our conditions on matrices and associated rates are essentially optimal.

Motivation & Objective

• To establish a restricted strong convexity (RSC) condition for the matrix completion problem under weighted, noisy sampling.
• To derive non-asymptotic error bounds in the weighted Frobenius norm for low-rank matrix recovery with noise.
• To relax the incoherence conditions used in prior work by introducing measures of matrix spikiness and low-rankness.
• To show that the proposed method achieves minimax-optimal rates up to logarithmic factors using information-theoretic lower bounds.
• To unify and generalize results for both uniform and non-uniform sampling, including reweighted nuclear norm regularization.

Proposed method

• Formalizes a weighted Frobenius norm and defines a matrix class $\mathfrak{C}$ with bounded $\ell_q$-spikiness and low-rank structure.
• Proposes an M-estimator that combines a data-fidelity term with a weighted nuclear norm regularizer to control rank and spikiness.
• Proves that the random observation operator satisfies restricted strong convexity (RSC) with high probability over the matrix class $\mathfrak{C}$.
• Employs matrix concentration inequalities, including the Ahlswede-Winter bound, to control the operator norm of noise terms.
• Uses a chaining argument and symmetrization to bound the expected supremum of a Rademacher chaos process.
• Derives non-asymptotic error bounds via RSC and decomposability of the regularizer, leading to optimal rates.

Experimental results

Research questions

• RQ1 Does the random observation operator for weighted matrix completion satisfy restricted strong convexity with high probability?
• RQ2 Can non-asymptotic error bounds be derived in the weighted Frobenius norm for noisy, approximately low-rank matrices?
• RQ3 Are the conditions on matrix spikiness and low-rankness less restrictive than incoherence conditions used in prior work?
• RQ4 Can the proposed method achieve minimax-optimal rates for matrix completion under noisy sampling?
• RQ5 How does the performance of the weighted nuclear norm estimator compare to standard nuclear norm in non-uniform sampling regimes?

Key findings

• The random observation operator satisfies restricted strong convexity with high probability over the matrix class $\mathfrak{C}$, enabling tight error bounds.
• The proposed M-estimator achieves error bounds of order $\mathcal{O}\big(\sqrt{\frac{d \log d}{n}}\big)$ in the weighted Frobenius norm under noise.
• The method achieves optimal rates up to logarithmic factors, as confirmed by information-theoretic lower bounds.
• The analysis relaxes incoherence assumptions by using $\ell_q$-spikiness and low-rankness measures, which are less restrictive.
• The error bounds hold for both uniform and non-uniform sampling, including reweighted nuclear norm regularization as a special case.
• The bound on the expected operator norm of the noise term is $\mathbb{E}[\|\frac{1}{n}\sum \varepsilon_i \widetilde{X}^{(i)}\|_2] \leq 10 \max\big\{\sqrt{\frac{L d \log d}{n}}, \frac{L d \log d}{n}\big\}$, which is used to derive the final error rate.