[Paper Review] Contrastive Learning for Recommender System

Zhuang Liu, Yunpu Ma+2 more|arXiv (Cornell University)|Jan 5, 2021

Recommender Systems and Techniques38 references44 citations

TL;DR

Introduces graph contrastive learning and a debiased contrastive loss to improve GNN-based recommender systems and mitigate selection and sampling biases; demonstrates improvements over multiple baselines across three datasets.

ABSTRACT

Recommender systems, which analyze users' preference patterns to suggest potential targets, are indispensable in today's society. Collaborative Filtering (CF) is the most popular recommendation model. Specifically, Graph Neural Network (GNN) has become a new state-of-the-art for CF. In the GNN-based recommender system, message dropout is usually used to alleviate the selection bias in the user-item bipartite graph. However, message dropout might deteriorate the recommender system's performance due to the randomness of dropping out the outgoing messages based on the user-item bipartite graph. To solve this problem, we propose a graph contrastive learning module for a general recommender system that learns the embeddings in a self-supervised manner and reduces the randomness of message dropout. Besides, many recommender systems optimize models with pairwise ranking objectives, such as the Bayesian Pairwise Ranking (BPR) based on a negative sampling strategy. However, BPR has the following problems: suboptimal sampling and sample bias. We introduce a new debiased contrastive loss to solve these problems, which provides sufficient negative samples and applies a bias correction probability to alleviate the sample bias. We integrate the proposed framework, including graph contrastive module and debiased contrastive module with several Matrix Factorization(MF) and GNN-based recommendation models. Experimental results on three public benchmarks demonstrate the effectiveness of our framework.

Motivation & Objective

• Motivate improving representation quality in GNN-based recommender systems by reducing selection bias from message dropout.
• Address suboptimal negative sampling and sampling bias in BPR-based optimization.
• Propose a general framework combining graph contrastive learning with a debiased contrastive loss.
• Show that the framework yields improvements across multiple baseline models and datasets.

Proposed method

• Propose a graph contrastive learning (GCL) module that perturbs L-hop subgraphs around users (and optionally items) and maximizes agreement between two perturbations.
• Introduce a debiased contrastive loss (DCL) that provides additional negative samples and a bias correction probability tau+ to reduce sampling bias.
• Combine L_GCL and L_DCL in an end-to-end objective with L2 regularization and train using Adam.
• Use cosine similarity with temperature parameters (t1, t2) in the contrastive losses and normalize embeddings.
• Treat other users and items in the same minibatch as negative samples to promote uniformity and separability in the embedding space.
• Apply the framework to various base recommender models (MF, DMF, GCN, GC-MC, PinSage, NGCF, LR-GCCF, LightGCN) by replacing their loss with GCL+DCL.

Experimental results

Research questions

• RQ1 RQ1: Is the proposed framework applicable to general and benchmark algorithms for recommender systems?
• RQ2 RQ2: What is the contribution of various components (graph perturbation, debiased loss) to performance?
• RQ3 RQ3: Is the framework more efficient than state-of-the-art models?
• RQ4 RQ4: How robust is the framework to hyperparameters like tau+ and graph perturbation p?

Key findings

• The framework consistently improves baseline models across Yelp2018, Amazon-Book, and Steam.
• Graph perturbation and debiased contrastive loss significantly boost performance for basic models like MF and DMF, with large relative gains on Yelp2018.
• State-of-the-art models like LightGCN and its variants also benefit, achieving top results in several datasets.
• MF/DMF show substantial gains when switching to GCL+DCL (e.g., MF++ gains up to ~60-67% on recall@20 and ndcg@20 in some datasets).
• GCN++, GC-MC++, LR-GCCF++, and LightGCN++ variants consistently outperform their base counterparts across datasets.