[Paper Review] Self Contrastive Learning for Session-based Recommendation

Zhengxiang Shi, Xi Wang+1 more|arXiv (Cornell University)|Jun 2, 2023

Recommender Systems and Techniques4 citations

TL;DR

This paper proposes Self Contrastive Learning (SCL), a simplified contrastive learning method that directly optimizes item representation uniformity without requiring positive/negative sample construction or data augmentation. SCL improves state-of-the-art session-based recommendation models by 8.2–11.2% in key metrics like P@10 and MRR@10, while reducing computational complexity through k-NN approximation.

ABSTRACT

Session-based recommendation, which aims to predict the next item of users' interest as per an existing sequence interaction of items, has attracted growing applications of Contrastive Learning (CL) with improved user and item representations. However, these contrastive objectives: (1) serve a similar role as the cross-entropy loss while ignoring the item representation space optimisation; and (2) commonly require complicated modelling, including complex positive/negative sample constructions and extra data augmentation. In this work, we introduce Self-Contrastive Learning (SCL), which simplifies the application of CL and enhances the performance of state-of-the-art CL-based recommendation techniques. Specifically, SCL is formulated as an objective function that directly promotes a uniform distribution among item representations and efficiently replaces all the existing contrastive objective components of state-of-the-art models. Unlike previous works, SCL eliminates the need for any positive/negative sample construction or data augmentation, leading to enhanced interpretability of the item representation space and facilitating its extensibility to existing recommender systems. Through experiments on three benchmark datasets, we demonstrate that SCL consistently improves the performance of state-of-the-art models with statistical significance. Notably, our experiments show that SCL improves the performance of two best-performing models by 8.2% and 9.5% in P@10 (Precision) and 9.9% and 11.2% in MRR@10 (Mean Reciprocal Rank) on average across different benchmarks. Additionally, our analysis elucidates the improvement in terms of alignment and uniformity of representations, as well as the effectiveness of SCL with a low computational cost.

Motivation & Objective

• To address the underutilization of item representation uniformity in contrastive learning-based session recommendation.
• To simplify existing complex contrastive learning objectives that rely on intricate positive/negative sample construction and data augmentation.
• To improve model performance through a unified, interpretable, and extensible contrastive objective focused solely on uniformity of item representations.
• To evaluate whether complex contrastive objectives remain necessary when SCL is applied.
• To demonstrate the efficiency and low computational cost of SCL through ablation and hyperparameter analysis.

Proposed method

• SCL introduces a novel contrastive loss that enforces uniformity across all item representations by contrasting each item's embedding against all others.
• The core objective is formulated as a self-contrastive loss: $\mathcal{L}^{\text{SCL}} = -\sum_{i=1}^{n} \log \frac{f(\mathbf{x}_i, \mathbf{x}_i)}{\sum_{\mathbf{x}' \in \mathcal{K}_i} f(\mathbf{x}_i, \mathbf{x}')} $, where $\mathcal{K}_i$ is the set of k-nearest neighbors of item $i$.
• To reduce computational cost, the method uses a k-Nearest Neighbor (kNN) approximation, reducing time complexity from $O(n^2 d)$ to $O(n k d)$.
• SCL is designed as a plug-in replacement for existing contrastive objectives in SOTA models, requiring no architectural changes or additional data augmentation.
• The method is compatible with any session-based recommendation model and enhances both alignment and uniformity of representations.
• The temperature hyperparameter $\tau$ is used to control the sharpness of the contrastive distribution, with optimal performance at $\tau = 0.1$.

Experimental results

Research questions

• RQ1 To what extent does SCL improve the performance of session-based recommendation models across different benchmarks?
• RQ2 How does SCL enhance model performance in terms of representation alignment and uniformity?
• RQ3 Are complex contrastive objectives still necessary when SCL is applied?
• RQ4 Can SCL maintain state-of-the-art performance with low computational cost?

Key findings

• SCL improves the performance of two SOTA models by 8.2% and 9.5% in P@10, and by 9.9% and 11.2% in MRR@10 across different benchmarks.
• SCL consistently improves model performance on three datasets: Tmall, Nowplaying, and Diginetica, with statistical significance.
• The model achieves optimal performance with a negative sample size $k = 2$ using the kNN approximation, indicating that complex negative sampling is unnecessary.
• The optimal temperature hyperparameter $\tau$ is found to be 0.1, with performance degrading at higher values.
• SCL maintains strong performance even when the full set of negative samples is used, but gains plateau beyond $k = 10$, showing diminishing returns.
• The kNN approximation reduces computational cost significantly, enabling efficient deployment without sacrificing performance.