[Paper Review] Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks

Federico Monti, Michael M. Bronstein+1 more|arXiv (Cornell University)|Apr 22, 2017

Graph Theory and AlgorithmsComputer Science241 citations

TL;DR

The paper introduces recurrent graph CNNs (RGCNN) for matrix completion, using multi-graph convolutions over user/item graphs coupled with an LSTM diffusion to predict ratings, achieving state-of-the-art results with a constant number of parameters.

ABSTRACT

Matrix completion models are among the most common formulations of recommender systems. Recent works have showed a boost of performance of these techniques when introducing the pairwise relationships between users/items in the form of graphs, and imposing smoothness priors on these graphs. However, such techniques do not fully exploit the local stationarity structures of user/item graphs, and the number of parameters to learn is linear w.r.t. the number of users and items. We propose a novel approach to overcome these limitations by using geometric deep learning on graphs. Our matrix completion architecture combines graph convolutional neural networks and recurrent neural networks to learn meaningful statistical graph-structured patterns and the non-linear diffusion process that generates the known ratings. This neural network system requires a constant number of parameters independent of the matrix size. We apply our method on both synthetic and real datasets, showing that it outperforms state-of-the-art techniques.

Motivation & Objective

• Motivate incorporating graph-structured priors into matrix completion to capture local stationarity in user/item graphs.
• Develop a neural architecture that learns graph-structured patterns and diffusion dynamics of ratings.
• Achieve constant parameter count with scalable factorized or full matrix formulations.
• Demonstrate superior performance on synthetic and real-world recommender datasets compared to baselines.

Proposed method

• Represent the matrix as living on row/column graphs with Laplacians for smoothness regularization on X. 4
• Use Multi-Graph CNNs (MGCNN) to extract spatial features across both user and item graphs via Chebyshev polynomial filters to maintain O(mn) complexity.
• Optionally employ separable (factorized) GCNNs on W and H to reduce parameters to O(m+n).
• Couple the spatial features with a Recurrent Neural Network (LSTM) to perform a diffusion-like temporal update X^(t+1)=X^(t)+dX^(t).
• Train end-to-end by minimizing a loss that includes graph-based regularization terms and a masked Frobenius fit to observed entries.
• Provide two architectures: (i) full matrix completion with MGCNN+RNN (RGCNN), and (ii) factorized separable model (sRGCNN) with two GCNNs + RNN for W and H.

Experimental results

Research questions

• RQ1 Can geometric deep learning on graphs improve matrix completion beyond traditional graph-regularized or low-rank methods?
• RQ2 Does a recurrent diffusion on graph-structured scores via MGCNNs outperform existing baselines on synthetic and real datasets?
• RQ3 What are the trade-offs between full matrix vs. factorized (W,H) representations in terms of scalability and accuracy?
• RQ4 How many diffusion steps and what graph-polynomial orders are needed to achieve robust performance?

Key findings

• On synthetic data with strong community structure, RGCNN achieves the best RMSE among methods considered (example: 0.0053 for RGCNN vs. higher values for GMC/GRALS/sRGCNN).
• In the synthetic-column-only setting, sRGCNN still outperforms GRALS and other baselines (e.g., RMSE 0.0362 vs. 0.0452 for GRALS).
• On MovieLens, RGCNN and its variants outperform baselines (e.g., sRGCNN RMSE 0.929 vs. MC 0.973 and GMC 0.996).
• Across Flixster, Douban, YahooMusic, the RGCNN family outperforms state-of-the-art methods like GRALS and GMC, with sRGCNN/RGCNN achieving the best reported RMSEs.
• The approach maintains a very small number of learned parameters (O(1) for full RGCNN; O(m+n) for separable sRGCNN) and scales with O(mn) or O(m+n) complexity depending on the architecture.
• The learned spectral filters (via Chebyshev polynomials) and the diffusion-based training produce interpretable spectral patterns and diffusion dynamics (illustrated in spectral filter plots).