[Paper Review] Neural Collaborative Filtering

Xiangnan He, Lizi Liao+4 more|arXiv (Cornell University)|Aug 16, 2017

Recommender Systems and TechniquesComputer Science34 references221 citations

TL;DR

The paper introduces Neural Collaborative Filtering (NCF), a neural-network-based framework to model user–item interactions for implicit feedback, unifying and extending matrix factorization with deep learning. It shows that GMF, MLP, and NeuMF improve recommendation accuracy on real datasets.

ABSTRACT

In recent years, deep neural networks have yielded immense success on speech recognition, computer vision and natural language processing. However, the exploration of deep neural networks on recommender systems has received relatively less scrutiny. In this work, we strive to develop techniques based on neural networks to tackle the key problem in recommendation -- collaborative filtering -- on the basis of implicit feedback. Although some recent work has employed deep learning for recommendation, they primarily used it to model auxiliary information, such as textual descriptions of items and acoustic features of musics. When it comes to model the key factor in collaborative filtering -- the interaction between user and item features, they still resorted to matrix factorization and applied an inner product on the latent features of users and items. By replacing the inner product with a neural architecture that can learn an arbitrary function from data, we present a general framework named NCF, short for Neural network-based Collaborative Filtering. NCF is generic and can express and generalize matrix factorization under its framework. To supercharge NCF modelling with non-linearities, we propose to leverage a multi-layer perceptron to learn the user-item interaction function. Extensive experiments on two real-world datasets show significant improvements of our proposed NCF framework over the state-of-the-art methods. Empirical evidence shows that using deeper layers of neural networks offers better recommendation performance.

Motivation & Objective

• Motivate the use of deep neural networks to model the user–item interaction function in collaborative filtering with implicit feedback.
• Propose a general neural collaborative filtering (NCF) framework that generalizes matrix factorization.
• Instantiate NCF with GMF and MLP, and propose NeuMF to fuse both.
• Demonstrate empirical gains on real-world datasets (MovieLens and Pinterest).
• Analyze the impact of network depth and pre-training on performance.

Proposed method

• Define user and item inputs as one-hot features mapped to embeddings.
• Model the interaction function f with neural networks (GMF as a linear kernel, MLP as a non-linear kernel, and NeuMF to fuse both).
• Train using probabilistic/logistic loss for implicit feedback (binary cross-entropy) with negative sampling.
• Provide a pre-training strategy for NeuMF by initializing from pre-trained GMF and MLP components.
• Compare against baseline MF-based methods and recommendations approaches on two datasets.

Experimental results

Research questions

• RQ1 Can neural networks capture complex user–item interactions beyond inner products in MF for implicit feedback?
• RQ2 Do deeper networks improve recommendation performance, and how do GMF, MLP, and NeuMF compare?
• RQ3 Does pre-training NeuMF from GMF and MLP improve optimization and results?
• RQ4 How do NC F variants perform relative to state-of-the-art baselines on real datasets?

Key findings

• NeuMF achieves the best performance, significantly outperforming eALS and BPR baselines on both datasets.
• GMF and MLP are strong individually, with NeuMF benefiting from their fusion.
• Pre-training NeuMF provides consistent gains across datasets, especially at higher factor sizes.
• NeuMF outperforms baselines in top-10 recommendation (HR@10 and NDCG@10) across K=1..10.
• Deeper networks (more predictive factors) generally improve performance, with diminishing returns in some cases.