[论文解读] Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction

Qi Pi, Xiaoqiang Zhu+6 more|arXiv (Cornell University)|Jun 10, 2020

Recommender Systems and Techniques参考文献 20被引用 5

一句话总结

本文提出搜索增强型兴趣建模（Search-based Interest Modeling, SIM），一种两阶段框架，首先通过通用搜索单元（General Search Unit）从长达54,000次交互的终身用户行为序列中提取相关子序列，随后通过精确搜索单元（Exact Search Unit）建模用户与物品之间的精确兴趣。SIM在阿里巴巴真实场景的展示广告系统中实现了7.1%的CTR提升和4.4%的RPM提升，显著优于先前方法在长期兴趣建模方面的表现，同时保持了低延迟。

ABSTRACT

Rich user behavior data has been proven to be of great value for click-through rate prediction tasks, especially in industrial applications such as recommender systems and online advertising. Both industry and academy have paid much attention to this topic and propose different approaches to modeling with long sequential user behavior data. Among them, memory network based model MIMN proposed by Alibaba, achieves SOTA with the co-design of both learning algorithm and serving system. MIMN is the first industrial solution that can model sequential user behavior data with length scaling up to 1000. However, MIMN fails to precisely capture user interests given a specific candidate item when the length of user behavior sequence increases further, say, by 10 times or more. This challenge exists widely in previously proposed approaches. In this paper, we tackle this problem by designing a new modeling paradigm, which we name as Search-based Interest Model (SIM). SIM extracts user interests with two cascaded search units: (i) General Search Unit acts as a general search from the raw and arbitrary long sequential behavior data, with query information from candidate item, and gets a Sub user Behavior Sequence which is relevant to candidate item; (ii) Exact Search Unit models the precise relationship between candidate item and SBS. This cascaded search paradigm enables SIM with a better ability to model lifelong sequential behavior data in both scalability and accuracy. Apart from the learning algorithm, we also introduce our hands-on experience on how to implement SIM in large scale industrial systems. Since 2019, SIM has been deployed in the display advertising system in Alibaba, bringing 7.1\% CTR and 4.4\% RPM lift, which is significant to the business. Serving the main traffic in our real system now, SIM models user behavior data with maximum length reaching up to 54000, pushing SOTA to 54x.

研究动机与目标

• 解决工业级点击率预测系统中建模超长序列用户行为数据（超过1,000次交互）的挑战。
• 在不牺牲推理延迟或系统可扩展性的前提下，提升长期兴趣建模的准确性。
• 实现实时推理系统中大规模用户行为序列的高效、可扩展且精确的用户兴趣建模。
• 在保留有意义的长期用户偏好同时，有效过滤长序列中的无关噪声行为。

提出的方法

• 通用搜索单元（GSU）利用候选物品特征，对原始、任意长度的用户行为序列执行轻量级、查询感知的搜索，筛选出约200个相关行为的子序列（SBS）。
• 精确搜索单元（ESU）采用类似于DIN或DIEN的注意力机制，建模候选物品与筛选后SBS之间的精确交互。
• 级联搜索设计将长序列过滤与细粒度兴趣建模解耦，支持可扩展性与准确性的权衡。
• 采用两阶段离线索引系统，按用户ID和交互类别组织用户行为数据，降低在线请求流量，实现高效检索。
• ESU中的多头注意力通过深度核融合进行优化，以维持低推理延迟。
• 该模型已部署于阿里巴巴的实时点击率预测系统中，支持长达54,000步的序列，仅比截断基线多出5ms延迟。

实验结果

研究问题

• RQ1 能否设计一种可扩展且准确的方法，在实时工业系统中对长达54,000步的用户行为序列进行建模？
• RQ2 如何有效过滤长序列行为数据中的无关噪声，同时保留长期用户兴趣？
• RQ3 级联搜索机制是否能优于固定大小记忆网络，在捕捉多样化、上下文相关的用户兴趣方面表现更优？
• RQ4 与现有SOTA方法（如MIMN和DIEN）相比，该模型在长期兴趣建模方面提升了多少？

主要发现

• 在2020年1月7日至2月7日期间的线上A/B测试中，SIM相较于MIMN实现了7.1%的点击率（CTR）提升和4.4%的千次展示收益（RPM）提升。
• 用户最后一次同类别行为的平均天数（Days till Last Same Category Behavior）从DIEN的11.2提升至SIM的13.3，表明长期用户兴趣建模能力更强。
• 用户在目标物品类别中存在历史行为的概率从DIEN的0.91提升至SIM的0.94，证实了长期偏好捕捉能力的提升。
• SIM成功将用户行为序列扩展至最长54,000步——是MIMN 1,000步限制的54倍——且未造成显著延迟增加。
• 系统保持了低推理延迟，在服务10,000步以上序列时，仅比MIMN的截断1,000步序列多出5ms延迟。
• 两阶段索引系统降低了在线请求流量，实现了高效检索，支持每秒超过100万用户的高吞吐量服务。