[Paper Review] DemiNet: Dependency-Aware Multi-Interest Network with Self-Supervised Graph Learning for Click-Through Rate Prediction

Yule Wang, Qiang Luo+4 more|arXiv (Cornell University)|Sep 26, 2021

Recommender Systems and Techniques27 references4 citations

TL;DR

DemiNet proposes a dependency-aware multi-interest network with self-supervised graph learning for click-through rate (CTR) prediction. It uses heterogeneous graph attention to model diverse item dependencies and denoise behavior sequences, applies self-supervised learning to enhance robustness in interest extraction, and employs route-specific experts with a confidence network for adaptive aggregation—achieving state-of-the-art performance on three real-world datasets.

ABSTRACT

In this paper, we propose a novel model named DemiNet (short for DEpendency-Aware Multi-Interest Network) to address the above two issues. To be specific, we first consider various dependency types between item nodes and perform dependency-aware heterogeneous attention for denoising and obtaining accurate sequence item representations. Secondly, for multiple interests extraction, multi-head attention is conducted on top of the graph embedding. To filter out noisy inter-item correlations and enhance the robustness of extracted interests, self-supervised interest learning is introduced to the above two steps. Thirdly, to aggregate the multiple interests, interest experts corresponding to different interest routes give rating scores respectively, while a specialized network assigns the confidence of each score. Experimental results on three real-world datasets demonstrate that the proposed DemiNet significantly improves the overall recommendation performance over several state-of-the-art baselines. Further studies verify the efficacy and interpretability benefits brought by the fine-grained user interest modeling.

Motivation & Objective

• To address noisy and intertwined user behavior sequences that hinder accurate multiple interest extraction in CTR prediction.
• To model latent correlations between extracted interest vectors, which are often neglected in prior methods.
• To improve robustness and interpretability of user interest representations through self-supervised learning.
• To enhance prediction accuracy by adaptively aggregating multiple interest representations using route-specific experts and a confidence network.

Proposed method

• Constructs a heterogeneous graph with four dependency types (contextual, similarity, sequential, and co-occurrence) to model item relationships in user behavior sequences.
• Applies hierarchical, dependency-aware heterogeneous attention to denoise sequences and generate accurate item representations.
• Introduces self-supervised interest learning via random edge dropout in both the interaction and multi-interest extraction modules to filter noisy correlations.
• Employs multi-head attention on graph embeddings to extract multiple core user interests.
• Uses route-specific interest experts to generate individual CTR predictions for each interest path.
• Deploys a Confi-Net to dynamically assign confidence weights to each expert's prediction, enabling adaptive aggregation.

Experimental results

Research questions

• RQ1 How can dependency-aware modeling of item relationships improve representation learning in noisy user behavior sequences for CTR prediction?
• RQ2 To what extent does self-supervised learning enhance the robustness of extracted multi-interest representations?
• RQ3 Can route-specific interest experts with confidence weighting improve prediction accuracy compared to single or concatenated interest vectors?
• RQ4 How do different graph convolution operators and hyperparameters like the number of interest routes affect model performance?

Key findings

• DemiNet achieves state-of-the-art performance on three real-world datasets, outperforming existing SOTA baselines in both AUC and Log Loss.
• The ablation study confirms that all components—DHA, self-supervised learning, and multi-interest aggregation—significantly improve model performance.
• The multi-dependency-aware heterogeneous attention (DHA) module yields the largest performance gain by effectively modeling contextual and similarity dependencies.
• Self-supervised interest learning enhances robustness by filtering out noisy inter-item correlations during interest extraction.
• The number of interest routes (K) shows diminishing returns beyond K=16, with performance saturating due to overfitting and increased computational cost.
• t-SNE visualization confirms that interest vectors from the same route cluster together, validating the model's ability to learn fine-grained, interpretable user intentions.