[Paper Review] A Riemannian gossip approach to decentralized matrix completion

Bamdev Mishra, Hiroyuki Kasai+1 more|arXiv (Cornell University)|May 23, 2016

Distributed Control Multi-Agent Systems21 references3 citations

TL;DR

This paper proposes a Riemannian gossip algorithm for decentralized low-rank matrix completion, where multiple agents collaboratively complete a matrix by minimizing a weighted sum of local completion and consensus terms on the Grassmann manifold. The method ensures asymptotic convergence, supports preconditioning for ill-conditioned problems, and achieves competitive performance on benchmarks including MovieLens 20M, outperforming D-LMaFit in scalability and accuracy.

ABSTRACT

In this paper, we propose novel gossip algorithms for the low-rank decentralized matrix completion problem. The proposed approach is on the Riemannian Grassmann manifold that allows local matrix completion by different agents while achieving asymptotic consensus on the global low-rank factors. The resulting approach is scalable and parallelizable. Our numerical experiments show the good performance of the proposed algorithms on various benchmarks.

Motivation & Objective

• To address the challenge of large-scale, decentralized matrix completion under privacy and communication constraints.
• To develop a gossip-based framework that enables agents to collaboratively complete a matrix while achieving consensus on global low-rank factors.
• To reduce communication overhead by restricting each agent to communicate with only one other agent at a time.
• To ensure asymptotic convergence of the algorithm under a decentralized, distributed setting.
• To improve performance on ill-conditioned and large-scale instances through preconditioning and parallelization.

Proposed method

• Formulates the decentralized matrix completion problem as a weighted sum of local completion and consensus terms on the Grassmann manifold.
• Uses fixed-rank matrix factorization with X = UW^T, where U ∈ St(r,m) and W ∈ R^{n×r}, to enforce low-rank structure.
• Employs stochastic Riemannian gradient descent on the Grassmann manifold to update U, with closed-form solution for W at each step.
• Implements a gossip-based communication protocol where agents exchange information with one neighbor per iteration to minimize communication.
• Proposes a preconditioned variant using a diagonal preconditioner to improve convergence on ill-conditioned problems.
• Introduces a parallel variant that enables simultaneous updates across agents, exploiting parallel computational architectures.

Experimental results

Research questions

• RQ1 Can a gossip-based decentralized framework achieve asymptotic consensus in low-rank matrix completion while minimizing communication?
• RQ2 How does the performance of the proposed Riemannian gossip algorithm compare to state-of-the-art methods like D-LMaFit on large-scale datasets?
• RQ3 To what extent does preconditioning improve convergence and accuracy in ill-conditioned matrix completion scenarios?
• RQ4 Can the proposed algorithm scale effectively to real-world datasets such as MovieLens 20M with billions of ratings?
• RQ5 How does the balance between completion and consensus terms, controlled by ρ, affect convergence and accuracy?

Key findings

• The Online Gossip algorithm outperforms D-LMaFit on the MovieLens 20M dataset, achieving a normalized mean absolute error (NMAE) of 0.1507 ± 3×10⁻³ at rank 5.
• The preconditioned variant significantly accelerates convergence on ill-conditioned problems, reducing training error faster than the non-preconditioned version.
• Asymptotic consensus is achieved among agents on the MovieLens 20M dataset, with inter-agent distance approaching zero over iterations.
• The parallel variant shows similar performance to the online variant on training and test sets, indicating effective scalability.
• For a 10,000×100,000 matrix of rank 5 and oversampling ratio 6, the algorithm achieves both completion and consensus when ρ=10³, validating the theoretical trade-off.
• The algorithm maintains robustness in dynamic settings, with agents 2 and 3 converging faster due to higher update frequency under assumption A1.