[Paper Review] When Multi-Level Meets Multi-Interest: A Multi-Grained Neural Model for Sequential Recommendation

Yu Tian, Jianxin Chang+3 more|arXiv (Cornell University)|May 3, 2022

Recommender Systems and Techniques4 citations

TL;DR

This paper proposes MGNM, a multi-grained neural model that unifies multi-level user preference modeling and multi-interest learning for sequential recommendation. By constructing user-aware item interaction graphs and applying iterative graph convolution, MGNM captures complex, hierarchical preferences, while a novel sequential capsule network enhances sequential pattern integration into multi-interest extraction, achieving state-of-the-art performance with gains up to 4.35% in HIT@5 and 3.12% in NDCG@5.

ABSTRACT

Sequential recommendation aims at identifying the next item that is preferred by a user based on their behavioral history. Compared to conventional sequential models that leverage attention mechanisms and RNNs, recent efforts mainly follow two directions for improvement: multi-interest learning and graph convolutional aggregation. Specifically, multi-interest methods such as ComiRec and MIMN, focus on extracting different interests for a user by performing historical item clustering, while graph convolution methods including TGSRec and SURGE elect to refine user preferences based on multi-level correlations between historical items. Unfortunately, neither of them realizes that these two types of solutions can mutually complement each other, by aggregating multi-level user preference to achieve more precise multi-interest extraction for a better recommendation. To this end, in this paper, we propose a unified multi-grained neural model(named MGNM) via a combination of multi-interest learning and graph convolutional aggregation. Concretely, MGNM first learns the graph structure and information aggregation paths of the historical items for a user. It then performs graph convolution to derive item representations in an iterative fashion, in which the complex preferences at different levels can be well captured. Afterwards, a novel sequential capsule network is proposed to inject the sequential patterns into the multi-interest extraction process, leading to a more precise interest learning in a multi-grained manner.

Motivation & Objective

• To address the limitation of existing sequential recommendation models that fail to model complex, multi-grained user preferences.
• To bridge the gap between multi-interest learning and multi-level preference modeling by unifying them in a single framework.
• To improve recommendation accuracy by capturing both high-level and fine-grained user interests through a learnable graph construction and iterative graph convolution process.
• To enhance multi-interest extraction by integrating sequential patterns via a novel sequential capsule network.
• To validate the effectiveness and robustness of the proposed model across diverse real-world recommendation scenarios.

Proposed method

• MGNM constructs a user-aware item interaction graph from historical user behaviors, transforming loose sequences into structured graphs for better preference modeling.
• It applies iterative graph convolution to derive item representations that capture complex, multi-level preferences across different granularities.
• A novel sequential capsule network is designed to inject sequential patterns into the multi-interest learning process, improving the precision of interest extraction.
• The model uses max-pooling over multiple graph convolution layers to identify the most relevant preference levels per user, enabling multi-grained interest representation.
• The framework integrates both multi-interest learning and multi-level preference modeling in an end-to-end trainable manner, with attention to user-specific interest dynamics.
• The model is trained using a contrastive learning objective to enhance representation quality and generalization across diverse user behaviors.

Experimental results

Research questions

• RQ1 Can combining multi-level preference modeling with multi-interest learning improve sequential recommendation performance?
• RQ2 How does the integration of sequential patterns into multi-interest extraction affect recommendation accuracy?
• RQ3 What is the impact of different levels of granularity in user interest representation on model performance?
• RQ4 How robust is the model across diverse recommendation scenarios with varying semantic complexity?
• RQ5 What is the optimal number of interest clusters (K) and graph convolution layers (L) for different datasets?

Key findings

• MGNM achieves state-of-the-art performance on three real-world datasets, with a 3.12% improvement in NDCG@5 and 4.35% in HIT@5 over the best baseline.
• The model demonstrates robustness across diverse recommendation scenarios, including e-commerce, micro-video, and music instruments, with stable performance across varying hyperparameter settings.
• The optimal number of interest clusters (K) is 2 for the Toys and Games dataset and 4 for the Micro-video dataset, reflecting differences in semantic complexity.
• Multi-level preference modeling significantly improves performance, especially on datasets with broader semantic spaces like Micro-video, where high-level preferences play a more prominent role.
• Max-pooling outperforms sum-pooling in the inference stage, confirming the effectiveness of the user-aware, multi-level preference selection mechanism.
• The training time is computationally efficient, requiring only about 1.5 hours per epoch on a single GPU, even for large-scale datasets like Micro-video.