[Paper Review] Outer Product-based Neural Collaborative Filtering

Xiangnan He, Xiaoyu Du+4 more|arXiv (Cornell University)|Aug 12, 2018

Recommender Systems and TechniquesComputer Science11 references55 citations

TL;DR

ConvNCF introduces an outer-product interaction map over user/item embeddings and uses CNNs to learn high-order correlations, achieving state-of-the-art top-k recommendations on Yelp and Gowalla implicit feedback data.

ABSTRACT

In this work, we contribute a new multi-layer neural network architecture named ONCF to perform collaborative filtering. The idea is to use an outer product to explicitly model the pairwise correlations between the dimensions of the embedding space. In contrast to existing neural recommender models that combine user embedding and item embedding via a simple concatenation or element-wise product, our proposal of using outer product above the embedding layer results in a two-dimensional interaction map that is more expressive and semantically plausible. Above the interaction map obtained by outer product, we propose to employ a convolutional neural network to learn high-order correlations among embedding dimensions. Extensive experiments on two public implicit feedback data demonstrate the effectiveness of our proposed ONCF framework, in particular, the positive effect of using outer product to model the correlations between embedding dimensions in the low level of multi-layer neural recommender model. The experiment codes are available at: https://github.com/duxy-me/ConvNCF

Motivation & Objective

• Motivate a neural collaborative filtering model that explicitly models correlations between embedding dimensions.
• Propose an outer product interaction map to capture pairwise embedding dimension relationships.
• Leverage CNNs to learn high-order correlations from the interaction map in a scalable way.
• Demonstrate empirical gains over state-of-the-art methods on implicit feedback datasets.

Proposed method

• Represent users and items with feature-based embeddings learned from one-hot encoded inputs.
• Compute an interaction map E as the outer product p_u q_i^T, forming a KxK matrix of pairwise dimension correlations.
• Apply a CNN over the interaction map to extract high-order correlation signals.
• Predict y_hat_ui via a final linear layer after CNN-derived features, trained with a Bayesian Personalized Ranking (BPR) objective.
• Pre-train embeddings with MF and fine-tune the ConvNCF model with L2 regularization on embeddings, CNN layers, and output layer.

Experimental results

Research questions

• RQ1 Can ConvNCF outperform state-of-the-art recommendation methods on implicit feedback data?
• RQ2 Do outer product interaction maps and CNN-based layers improve learning of dimension correlations over traditional MLP/GMF approaches?
• RQ3 How does the number of convolutional feature maps affect ConvNCF performance?
• RQ4 Is ConvNCF more stable and scalable than MLP-based counterparts for this task?

Key findings

• ConvNCF generally achieves the best Top-k performance across Yelp and Gowalla datasets compared with baselines.
• Replacing the outer product with concatenation or element-wise product reduces performance, showing the benefit of modeling dimension correlations.
• CNN-based ConvNCF (ConvNCF) outperforms an equivalent CNN-on-MLP (ONCF-mlp), indicating the efficiency and effectiveness of the local connectivity design.
• JRL (MLP-based) is consistently outperformed by ConvNCF, highlighting the importance of explicit embedding-dimension correlations.
• The number of CNN feature maps per layer can vary without dramatic loss in performance, indicating good generalization of the ConvNCF architecture.