[论文解读] Rethinking Lifelong Sequential Recommendation with Incremental Multi-Interest Attention

Yongji Wu, Lu Yin+5 more|arXiv (Cornell University)|May 28, 2021

Recommender Systems and Techniques参考文献 39被引用 6

一句话总结

本文提出LimaRec，一种终身序列推荐模型，采用增量多兴趣自注意力机制，高效捕捉来自长行为序列的多样化用户兴趣。通过用自注意力机制替代基于RNN的记忆网络，LimaRec在在线推理中实现恒定时间复杂度，同时在四个真实世界数据集上优于最先进基线模型，尤其在长序列用户上表现更优。

ABSTRACT

Sequential recommendation plays an increasingly important role in many e-commerce services such as display advertisement and online shopping. With the rapid development of these services in the last two decades, users have accumulated a massive amount of behavior data. Richer sequential behavior data has been proven to be of great value for sequential recommendation. However, traditional sequential models fail to handle users' lifelong sequences, as their linear computational and storage cost prohibits them from performing online inference. Recently, lifelong sequential modeling methods that borrow the idea of memory networks from NLP are proposed to address this issue. However, the RNN-based memory networks built upon intrinsically suffer from the inability to capture long-term dependencies and may instead be overwhelmed by the noise on extremely long behavior sequences. In addition, as the user's behavior sequence gets longer, more interests would be demonstrated in it. It is therefore crucial to model and capture the diverse interests of users. In order to tackle these issues, we propose a novel lifelong incremental multi-interest self attention based sequential recommendation model, namely LimaRec. Our proposed method benefits from the carefully designed self-attention to identify relevant information from users' behavior sequences with different interests. It is still able to incrementally update users' representations for online inference, similarly to memory network based approaches. We extensively evaluate our method on four real-world datasets and demonstrate its superior performances compared to the state-of-the-art baselines.

研究动机与目标

• 解决基于RNN的终身序列推荐模型存在的长期依赖问题和高计算成本问题。
• 在长行为序列中建模多样化、多维度的用户兴趣，同时避免被噪声淹没。
• 在严格延迟和存储约束下，实现恒定时间在线推理，适用于实时推荐系统。
• 通过捕捉不同兴趣模式，提升具有长而复杂行为历史用户的推荐性能。

提出的方法

• 利用自注意力机制从终身序列中动态检索相关行为，减少无关项目带来的噪声影响。
• 采用增量自注意力模块，以恒定时间更新用户表征，支持在线学习。
• 引入多兴趣提取模块，通过在行为序列上进行软搜索，为每个用户学习K个独立的兴趣向量。
• 使用两个堆叠的增量自注意力模块，优化用户表征并增强对时间动态的建模能力。
• 应用基于查询的注意力机制，使每个兴趣向量作为查询，检索相关的历史行为。
• 维护用户行为序列的个性化记忆，随每次新交互增量更新。

实验结果

研究问题

• RQ1 自注意力机制能否在保持长期依赖的前提下，有效替代基于RNN的记忆网络用于终身序列推荐？
• RQ2 建模多个用户兴趣在长行为序列上如何提升推荐准确性？
• RQ3 增量自注意力在降低计算与存储成本的同时，性能下降程度如何？
• RQ4 该模型在具有极长序列的用户上表现如何，尤其是在序列存在噪声或稀疏的情况下？

主要发现

• LimaRec在四个真实世界数据集（包括MovieLens和工业级电商数据）上均优于最先进基线模型。
• 在长行为序列用户上，LimaRec的性能下降速度明显慢于基线模型，表明其对序列长度具有更强鲁棒性。
• 随着兴趣向量数量K的增加，命中率显著提升，峰值出现在K=9，证明多兴趣建模的重要性。
• 推荐多样性随K增加而提升，但因K与用户真实兴趣数量不匹配，出现一定波动。
• 即使在数据稀疏或时间上不连续的情况下（如MovieLens中长达数年的用户历史），模型在长序列上仍保持高性能。