[Paper Review] Deep Interest Evolution Network for Click-Through Rate Prediction

Guorui Zhou, Na Mou+6 more|arXiv (Cornell University)|Sep 11, 2018

Recommender Systems and Techniques68 citations

TL;DR

DIEN introduces an interest extractor with auxiliary supervision and an interest-evolving layer using AUGRU to capture target-item-relative interest evolution, achieving state-of-the-art CTR prediction and notable online gains.

ABSTRACT

Click-through rate~(CTR) prediction, whose goal is to estimate the probability of the user clicks, has become one of the core tasks in advertising systems. For CTR prediction model, it is necessary to capture the latent user interest behind the user behavior data. Besides, considering the changing of the external environment and the internal cognition, user interest evolves over time dynamically. There are several CTR prediction methods for interest modeling, while most of them regard the representation of behavior as the interest directly, and lack specially modeling for latent interest behind the concrete behavior. Moreover, few work consider the changing trend of interest. In this paper, we propose a novel model, named Deep Interest Evolution Network~(DIEN), for CTR prediction. Specifically, we design interest extractor layer to capture temporal interests from history behavior sequence. At this layer, we introduce an auxiliary loss to supervise interest extracting at each step. As user interests are diverse, especially in the e-commerce system, we propose interest evolving layer to capture interest evolving process that is relative to the target item. At interest evolving layer, attention mechanism is embedded into the sequential structure novelly, and the effects of relative interests are strengthened during interest evolution. In the experiments on both public and industrial datasets, DIEN significantly outperforms the state-of-the-art solutions. Notably, DIEN has been deployed in the display advertisement system of Taobao, and obtained 20.7\% improvement on CTR.

Motivation & Objective

• Motivation to improve CTR prediction by modeling latent, evolving user interests behind explicit behaviors.
• Propose a dedicated interest extractor to obtain expressive latent interests from history actions.
• Model the dynamic evolution of interests in relation to the target item.
• Demonstrate effectiveness on public datasets and industrial data, including online deployment.
• Show that auxiliary supervision and AUGRU improve representation and prediction.

Proposed method

• Use GRU-based interest extractor to capture temporal dependencies in user behavior sequences.
• Introduce auxiliary loss that supervises hidden states with next-behavior information to enrich latent interests.
• Propose interest evolving layer with AUGRU to model evolution relative to the target item, using attention to modulate influence.
• Incorporate attention a_t to compute relevance between historical interests and target-item embedding; augment GRU with attentional update gate (AUGRU).
• Train with target CTR loss plus auxiliary loss, balancing via parameter alpha.

Experimental results

Research questions

• RQ1 Can latent latent interests extracted from history improve CTR beyond direct behavior representations?
• RQ2 Does modeling interest evolution relative to the target item improve CTR prediction and robustness to interest drifting?
• RQ3 Does auxiliary supervision of GRU hidden states enhance the quality of interest representations?
• RQ4 How does AUGRU compare to other attention/GRU variants in capturing evolving user interests?
• RQ5 What are the practical online performance gains when deploying DIEN in a large-scale ad system?

Key findings

• DIEN achieves higher AUC than baselines on public datasets Electronics and Books (0.7792 vs 0.7435 and 0.8453 vs 0.7686 for Electronics and Books, respectively).
• DIEN outperforms baselines on industrial dataset with AUC of 0.6541, surpassing DIN and GRU-based variants.
• AUGRU-based interest evolving significantly improves CTR prediction over AIGRU and AGRU variants.
• Auxiliary loss improves interest representation and embedding learning, with visible gains on public datasets and notable, though smaller, gains on industrial data.
• Online deployment in Taobao display ads yielded a 20.7% CTR gain and 17.1% eCPM gain vs a BaseModel baseline (pointers to real-world impact).
• Visualization and ablation studies show AUGRU effectively concentrates evolving interests around the target item, reducing interference from interest drifting.