[Paper Review] Deep Session Interest Network for Click-Through Rate Prediction

Yufei Feng, Fuyu Lv+5 more|arXiv (Cornell University)|May 16, 2019

Recommender Systems and Techniques19 references39 citations

TL;DR

DSIN models user behavior as multiple historical sessions, extracting per-session interests with self-attention, then modeling their evolution with Bi-LSTM and aggregating with a local activation unit to predict CTR.

ABSTRACT

Click-Through Rate (CTR) prediction plays an important role in many industrial applications, such as online advertising and recommender systems. How to capture users' dynamic and evolving interests from their behavior sequences remains a continuous research topic in the CTR prediction. However, most existing studies overlook the intrinsic structure of the sequences: the sequences are composed of sessions, where sessions are user behaviors separated by their occurring time. We observe that user behaviors are highly homogeneous in each session, and heterogeneous cross sessions. Based on this observation, we propose a novel CTR model named Deep Session Interest Network (DSIN) that leverages users' multiple historical sessions in their behavior sequences. We first use self-attention mechanism with bias encoding to extract users' interests in each session. Then we apply Bi-LSTM to model how users' interests evolve and interact among sessions. Finally, we employ the local activation unit to adaptively learn the influences of various session interests on the target item. Experiments are conducted on both advertising and production recommender datasets and DSIN outperforms other state-of-the-art models on both datasets.

Motivation & Objective

• Highlight that user behavior sequences are composed of sessions with homogeneous within-session behavior and heterogeneous cross-session behavior.
• Propose DSIN to leverage multiple historical sessions for CTR prediction.
• Develop a session-level interest extractor using self-attention with bias encoding.
• Model inter-session evolution with Bi-LSTM to capture sequential relationships among session interests.
• Aggregate session interests adaptively with a local activation unit before final prediction.

Proposed method

• Divide user behavior sequences into sessions based on time gaps (30 minutes rule).
• Apply bias-encoded self-attention to extract per-session interests.
• Use Bi-LSTM to capture sequential interactions among session interests.
• Employ a local activation unit to adaptively weigh session interests against the target item.
• Concatenate session-aware representations with user/item embeddings and pass through an MLP to predict CTR.

Experimental results

Research questions

• RQ1 Can leveraging multiple historical sessions with session-level attention improve CTR prediction over single-sequence models?
• RQ2 Do inter-session evolution and adaptive aggregation of session interests yield better predictive performance?
• RQ3 What is the contribution of bias encoding and session-interaction modeling to CTR accuracy?

Key findings

• DSIN achieves the best AUC on both advertising and recommender datasets compared to baselines.
• Incorporating multiple sessions and modeling their evolution improves performance over single-session models like DIN/DIEN.
• The bias encoding and session-interacting layer significantly enhance DSIN’s effectiveness.
• The local activation unit provides adaptive weighting of session interests with respect to the target item.