[Paper Review] Matrix Completion on Graphs

Vassilis Kalofolias, Xavier Bresson+2 more|arXiv (Cornell University)|Aug 7, 2014

Sparse and Compressive Sensing Techniques22 references147 citations

TL;DR

This paper proposes a novel matrix completion method that integrates graph-structured similarity information for rows and columns into a low-rank matrix recovery framework. By enforcing smoothness on user and item graphs via Laplacian regularization, the method improves reconstruction accuracy—especially under sparse observations—outperforming standard nuclear norm minimization in both synthetic and real-world recommendation datasets like Movielens-10M.

ABSTRACT

The problem of finding the missing values of a matrix given a few of its entries, called matrix completion, has gathered a lot of attention in the recent years. Although the problem under the standard low rank assumption is NP-hard, Candès and Recht showed that it can be exactly relaxed if the number of observed entries is sufficiently large. In this work, we introduce a novel matrix completion model that makes use of proximity information about rows and columns by assuming they form communities. This assumption makes sense in several real-world problems like in recommender systems, where there are communities of people sharing preferences, while products form clusters that receive similar ratings. Our main goal is thus to find a low-rank solution that is structured by the proximities of rows and columns encoded by graphs. We borrow ideas from manifold learning to constrain our solution to be smooth on these graphs, in order to implicitly force row and column proximities. Our matrix recovery model is formulated as a convex non-smooth optimization problem, for which a well-posed iterative scheme is provided. We study and evaluate the proposed matrix completion on synthetic and real data, showing that the proposed structured low-rank recovery model outperforms the standard matrix completion model in many situations.

Motivation & Objective

• To improve low-rank matrix completion by incorporating structural information from user and item proximity graphs.
• To address the limitations of standard matrix completion under sparse, non-uniformly sampled observations.
• To unify collaborative filtering (low-rank assumption) and content-based filtering (graph-based similarities) into a single convex optimization framework.
• To develop a robust, scalable algorithm that leverages manifold smoothness for better generalization in recommendation systems.
• To validate the method on synthetic and real-world data, demonstrating superiority in low-observation regimes.

Proposed method

• The method formulates matrix completion as a convex optimization problem combining nuclear norm minimization and graph Laplacian regularization.
• It enforces smoothness of the recovered matrix on user and item graphs using Dirichlet energy (Laplacian-based regularization).
• The optimization is solved via an alternating direction method of multipliers (ADMM) scheme, alternating between nuclear norm proximal updates and linear system solves.
• Graphs are constructed using exponential kernel-based similarity functions on user and item features, with adaptive neighborhood selection via epsilon-neighborhood or k-NN.
• The model uses a transfer function to weight graph edges based on distance, ensuring only relevant proximities contribute to regularization.
• Parameter selection is performed via 5-fold cross-validation on training sets of varying size, with performance evaluated on a fixed test set.

Experimental results

Research questions

• RQ1 Can incorporating graph-structured similarity between users and items improve matrix completion performance under sparse observations?
• RQ2 How does the proposed graph-regularized low-rank model compare to standard nuclear norm minimization in terms of reconstruction error?
• RQ3 Does the model remain robust when graphs are imperfect or non-uniformly sampled?
• RQ4 Under what observation sparsity levels does the graph regularization provide the most benefit?
• RQ5 Can the method effectively combine collaborative and content-based filtering into a unified optimization framework?

Key findings

• The proposed method with joint nuclear norm and graph regularization (red line) outperforms both standard nuclear norm (blue line) and graph-only regularization (green line) across most observation levels on Movielens-10M data.
• At 1% observed entries, graph-only regularization performs best, indicating that the nuclear norm term may be less beneficial when data is extremely sparse.
• At 32% observed entries, the standard nuclear norm method matches the performance of the combined model, suggesting diminishing returns of graph regularization at high data density.
• The reconstruction error of the proposed model is below 1.1 star RMS on Movielens-10M, indicating high accuracy in predicting user ratings.
• The model is robust to non-uniform sampling and imperfect graph construction, maintaining strong performance even with suboptimal graphs.
• The method significantly outperforms standard matrix completion when the number of observations is low, demonstrating its value in real-world recommendation systems with limited user ratings.