[Paper Review] Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives

Da Xu, Chuanwei Ruan+3 more|arXiv (Cornell University)|Oct 23, 2021

Domain Adaptation and Few-Shot Learning54 references4 citations

TL;DR

This paper provides a rigorous theoretical analysis of neural collaborative filtering (NCF) and matrix factorization collaborative filtering (MCF), showing both models converge to similar norm-constrained max-margin solutions under gradient descent. It reveals that MCF generalizes better in transductive settings, while NCF suffers from exposure bias in inductive settings, with a novel generalization bound accounting for data missing-not-at-random distributions.

ABSTRACT

The recent work by Rendle et al. (2020), based on empirical observations, argues that matrix-factorization collaborative filtering (MCF) compares favorably to neural collaborative filtering (NCF), and conjectures the dot product's superiority over the feed-forward neural network as similarity function. In this paper, we address the comparison rigorously by answering the following questions: 1. what is the limiting expressivity of each model; 2. under the practical gradient descent, to which solution does each optimization path converge; 3. how would the models generalize under the inductive and transductive learning setting. Our results highlight the similar expressivity for the overparameterized NCF and MCF as kernelized predictors, and reveal the relation between their optimization paths. We further show their different generalization behaviors, where MCF and NCF experience specific tradeoff and comparison in the transductive and inductive collaborative filtering setting. Lastly, by showing a novel generalization result, we reveal the critical role of correcting exposure bias for model evaluation in the inductive setting. Our results explain some of the previously observed conflicts, and we provide synthetic and real-data experiments to shed further insights to this topic.

Motivation & Objective

• To resolve the longstanding debate on whether neural collaborative filtering (NCF) outperforms matrix factorization collaborative filtering (MCF) by providing a rigorous theoretical foundation.
• To investigate the limiting expressivity of NCF and MCF as kernelized predictors in overparameterized regimes.
• To analyze the inductive bias of gradient descent optimization paths in both models and determine their convergence behavior.
• To compare generalization performance under transductive and inductive learning settings, particularly under data missing-not-at-random (MNAR) conditions.
• To derive a novel generalization bound that accounts for exposure bias in inductive collaborative filtering, improving model evaluation reliability.

Proposed method

• Leverages the Neural Tangent Kernel (NTK) framework to analyze the limiting expressivity of NCF and MCF as kernelized predictors.
• Derives a unified optimization analysis showing both NCF and MCF converge to norm-constrained max-margin solutions under gradient descent.
• Introduces a provably tight norm-constraint capacity analysis to compare generalization behavior in transductive and inductive settings.
• Proposes an inverse propensity weighting (IPW) correction mechanism to account for exposure bias in inductive learning, improving generalization evaluation.
• Employs synthetic and real-world data experiments to validate theoretical findings, including AUC comparisons under biased and unbiased evaluation metrics.
• Uses convex optimization (CVXOPT) to compute exact nuclear-norm max-margin solutions for comparison with MCF and NCF predictions.

Experimental results

Research questions

• RQ1 What is the limiting expressivity of NCF and MCF as kernelized predictors in the overparameterized regime?
• RQ2 Under gradient descent optimization, to which solution do NCF and MCF converge, and how do their optimization paths compare?
• RQ3 How do NCF and MCF generalize under transductive versus inductive learning settings?
• RQ4 What role does exposure bias play in model evaluation, and how can it be corrected to improve generalization assessment in inductive settings?
• RQ5 How does the discrepancy between empirical and exposure distributions affect model generalization, and can it be theoretically bounded?

Key findings

• Both NCF and MCF converge to norm-constrained max-margin solutions under gradient descent, indicating a shared inductive bias despite architectural differences.
• The expressivity of overparameterized NCF and MCF is equivalent when viewed as kernelized predictors, with both modeling the same class of functions in the infinite-width limit.
• MCF generalizes better than NCF in transductive settings due to its inherent invariance to data distribution shifts, while NCF underperforms due to exposure bias.
• In inductive settings, NCF exhibits poor generalization when exposure bias is uncorrected, but performance improves significantly with inverse propensity weighting (IPW) correction.
• A novel generalization bound is derived that accounts for data missing-not-at-random (MNAR) mechanisms, showing that exposure bias correction is critical for reliable evaluation.
• Experiments confirm that NCF with IPW achieves unbiased AUC performance comparable to MCF, resolving prior empirical contradictions in model comparison.