[Paper Review] Contrastive Self-supervised Sequential Recommendation with Robust Augmentation

Zhiwei Liu, Yongjun Chen+4 more|arXiv (Cornell University)|Aug 14, 2021

Recommender Systems and Techniques57 references83 citations

TL;DR

The paper introduces CoSeRec, a contrastive self-supervised framework for sequential recommendation that uses informative sequence augmentations to improve robustness to data sparsity and noise, jointly training with next-item prediction.

ABSTRACT

Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. At their core, such approaches model transition probabilities between items in a sequence, whether through Markov chains, recurrent networks, or more recently, Transformers. However both old and new issues remain, including data-sparsity and noisy data; such issues can impair the performance, especially in complex, parameter-hungry models. In this paper, we investigate the application of contrastive Self-Supervised Learning (SSL) to the sequential recommendation, as a way to alleviate some of these issues. Contrastive SSL constructs augmentations from unlabelled instances, where agreements among positive pairs are maximized. It is challenging to devise a contrastive SSL framework for a sequential recommendation, due to its discrete nature, correlations among items, and skewness of length distributions. To this end, we propose a novel framework, Contrastive Self-supervised Learning for sequential Recommendation (CoSeRec). We introduce two informative augmentation operators leveraging item correlations to create high-quality views for contrastive learning. Experimental results on three real-world datasets demonstrate the effectiveness of the proposed method on improving model performance and the robustness against sparse and noisy data. Our implementation is available online at \url{https://github.com/YChen1993/CoSeRec}

Motivation & Objective

• Motivate and address data-sparsity and noisy interactions in sequential recommendation (SR).
• Introduce informative sequence augmentations that preserve item correlations and handle length skewness.
• Propose a contrastive SSL objective and a multi-task training strategy to jointly optimize SR and SSL.

Proposed method

• Propose two informative augmentation operators— Substitute (S) and Insert (I)—that leverage item correlations to create robust views.
• Incorporate both memory-based and model-based item correlations and fuse them into a hybrid Cor_h for augmentation selection.
• Adopt three random augmentations (Crop, Mask, Reorder) and tailor augmentation sets by sequence length to support short sequences.
• Use NT-Xent contrastive loss to maximize agreement between two augmented views of the same sequence.
• Train the SR encoder with a multi-task objective combining next-item prediction loss and the ssl loss, with a balancing hyper-parameter lambda.

Experimental results

Research questions

• RQ1 How does CoSeRec perform in SR compared to existing methods?
• RQ2 Which augmentation strategies best suit SR, and how do informative augmentations impact performance?
• RQ3 Can CoSeRec robustly handle sparse data and noisy interactions in SR?
• RQ4 How do different hyperparameters and augmentation configurations affect CoSeRec?

Key findings

• CoSeRec consistently outperforms baseline models across three real-world datasets on HR@k and NDCG@k metrics.
• Informative augmentations that leverage item correlations improve positive view quality and mitigate cold-start issues, especially for short sequences.
• Joint multi-task training (SR + SSL) yields better performance than two-stage pre-training followed by SR fine-tuning.
• Hybrid correlation (Cor_h) for selecting correlated items improves augmentation robustness as training progresses.