[Paper Review] Matrix Completion from Non-Uniformly Sampled Entries

Yuanyu Wan, Jinfeng Yi+1 more|arXiv (Cornell University)|Jun 27, 2018

Sparse and Compressive Sensing Techniques17 references3 citations

TL;DR

This paper proposes a matrix completion algorithm that recovers low-rank matrices from non-uniformly sampled entries, where some columns are fully observed and others are partially observed with uniform sampling. It first recovers the column space from fully observed columns and then reconstructs each partially observed column by projecting onto this space, achieving exact recovery with O(rn ln n) entries and providing a Frobenius norm error bound for noisy matrices.

ABSTRACT

In this paper, we consider matrix completion from non-uniformly sampled entries including fully observed and partially observed columns. Specifically, we assume that a small number of columns are randomly selected and fully observed, and each remaining column is partially observed with uniform sampling. To recover the unknown matrix, we first recover its column space from the fully observed columns. Then, for each partially observed column, we recover it by finding a vector which lies in the recovered column space and consists of the observed entries. When the unknown $m imes n$ matrix is low-rank, we show that our algorithm can exactly recover it from merely $Ω(rn\ln n)$ entries, where $r$ is the rank of the matrix. Furthermore, for a noisy low-rank matrix, our algorithm computes a low-rank approximation of the unknown matrix and enjoys an additive error bound measured by Frobenius norm. Experimental results on synthetic datasets verify our theoretical claims and demonstrate the effectiveness of our proposed algorithm.

Motivation & Objective

• To address matrix completion in scenarios where entries are sampled non-uniformly, particularly when entire columns are fully observed and others are partially observed.
• To develop a recovery algorithm that works under more general sampling assumptions than prior work, including cases where only some columns are fully observed.
• To establish theoretical guarantees for exact recovery of low-rank matrices and for low-rank approximation under noise.
• To demonstrate through experiments that the proposed method outperforms existing baselines like Nyström in terms of accuracy and robustness.

Proposed method

• The algorithm first recovers the column space of the unknown matrix using a set of fully observed columns.
• For each partially observed column, it finds a vector in the recovered column space that matches the observed entries.
• The method relies on the incoherence condition of the matrix and uniform sampling of entries within partially observed columns.
• It uses singular value decomposition (SVD) to estimate the column space and projection to reconstruct missing entries.
• For noisy matrices, the algorithm computes a low-rank approximation and bounds the error using the Frobenius norm.
• Theoretical analysis leverages concentration inequalities and union bounds to derive sample complexity bounds.

Experimental results

Research questions

• RQ1 Can a low-rank matrix be exactly recovered when only a subset of columns is fully observed and the rest are partially observed with uniform sampling?
• RQ2 What is the minimum number of observed entries required for exact recovery under this non-uniform sampling model?
• RQ3 How does the proposed algorithm perform in the presence of noise, and what error bounds can be established?
• RQ4 Can the algorithm achieve better sample complexity than conventional matrix completion under uniform sampling?
• RQ5 How does the algorithm compare to existing methods like Nyström and CUR+ in terms of approximation accuracy?

Key findings

• The algorithm achieves exact recovery of a low-rank matrix with O(rn ln n) observed entries, where r is the matrix rank and n is the number of columns.
• The sample complexity is slightly better than the conventional matrix completion bound of O(rn ln²n) under uniform sampling.
• For noisy low-rank matrices, the algorithm provides a Frobenius norm error bound that scales with the noise level and matrix incoherence.
• Experimental results show that the algorithm outperforms the Nyström method in low-rank approximation, even when both observe the same number of entries.
• The required number of observed entries per partially observed column (s) scales linearly with r ln r rather than r² ln r, confirming the theoretical bound.
• The minimum number of fully observed columns (d) also scales linearly with r ln r, indicating that the sample complexity is dominated by the partially observed columns.