[Paper Review] Sequential Recommendation with Graph Neural Networks

Jianxin Chang, Chen Gao+6 more|arXiv (Cornell University)|Jun 27, 2021

Recommender Systems and TechniquesComputer Science62 references20 citations

TL;DR

This paper proposes SURGE, a graph neural network model that transforms loose user behavior sequences into tight item-item interest graphs to distill core preferences from noisy implicit feedback. By applying cluster-aware graph convolution and dynamic graph pooling, SURGE effectively captures evolving user interests, achieving state-of-the-art performance on both public and industrial datasets, especially in long-sequence modeling.

ABSTRACT

Sequential recommendation aims to leverage users' historical behaviors to predict their next interaction. Existing works have not yet addressed two main challenges in sequential recommendation. First, user behaviors in their rich historical sequences are often implicit and noisy preference signals, they cannot sufficiently reflect users' actual preferences. In addition, users' dynamic preferences often change rapidly over time, and hence it is difficult to capture user patterns in their historical sequences. In this work, we propose a graph neural network model called SURGE (short for SeqUential Recommendation with Graph neural nEtworks) to address these two issues. Specifically, SURGE integrates different types of preferences in long-term user behaviors into clusters in the graph by re-constructing loose item sequences into tight item-item interest graphs based on metric learning. This helps explicitly distinguish users' core interests, by forming dense clusters in the interest graph. Then, we perform cluster-aware and query-aware graph convolutional propagation and graph pooling on the constructed graph. It dynamically fuses and extracts users' current activated core interests from noisy user behavior sequences. We conduct extensive experiments on both public and proprietary industrial datasets. Experimental results demonstrate significant performance gains of our proposed method compared to state-of-the-art methods. Further studies on sequence length confirm that our method can model long behavioral sequences effectively and efficiently.

Motivation & Objective

• To address the challenge of modeling dynamic and evolving user preferences from long, noisy behavior sequences in sequential recommendation.
• To overcome the limitations of existing methods that focus heavily on recent behaviors and fail to effectively utilize long-term, implicit feedback.
• To propose a graph-based framework that explicitly clusters and extracts users' core interests from implicit signals.
• To enable effective and efficient modeling of long behavioral sequences by transforming them into structured interest graphs.

Proposed method

• Construct a tight item-item interest graph by re-encoding loose user behavior sequences using metric learning to group similar items based on user interaction patterns.
• Apply an attentive graph convolutional network to aggregate weak preference signals into stronger, more representative interest clusters.
• Introduce cluster-aware graph convolution to propagate and refine user interest representations within each cluster.
• Implement dynamic graph pooling that adaptively selects and preserves only the currently activated core interests based on query-aware attention.
• Use a learnable attention mechanism to weight the importance of different interest clusters at inference time, reflecting current user preferences.
• Train the model end-to-end to jointly optimize interest clustering and next-item prediction.

Experimental results

Research questions

• RQ1 Can a graph-based approach effectively distill core user interests from long, noisy behavior sequences with implicit feedback?
• RQ2 How well can a graph neural network model capture the dynamic evolution of user preferences over time?
• RQ3 Does transforming loose behavior sequences into tight interest graphs improve sequential recommendation performance compared to existing methods?
• RQ4 Can the proposed method effectively model long behavioral sequences without performance degradation?
• RQ5 How does dynamic graph pooling contribute to filtering out inactive or irrelevant interests during prediction?

Key findings

• SURGE achieves significant performance gains over state-of-the-art methods on both public and proprietary industrial datasets, with improvements in both recall and precision metrics.
• On the public DLRM dataset, SURGE achieves a recall@20 of 0.8906 and precision@20 of 0.4228, outperforming the previous SOTA.
• On the proprietary Kuaishou dataset, SURGE achieves a recall@20 of 0.8525 and precision@20 of 0.9316, demonstrating strong generalization on real-world data.
• Ablation studies confirm that the graph construction and dynamic pooling components are critical for performance, especially in long-sequence scenarios.
• The model maintains high efficiency and scalability, showing stable performance even as sequence length increases.
• The method effectively filters out noisy, non-core preferences, focusing on currently activated interests, which leads to more accurate next-item predictions.