[论文解读] Enhancing Bandit Algorithms with LLMs for Time-varying User Preferences in Streaming Recommendations

Chenglei Shen, Yi Zhan+3 more|arXiv (Cornell University)|Feb 8, 2026

Advanced Bandit Algorithms Research被引用 0

一句话总结

HyperBandit+ 引入时间感知的超网络和基于LLM的暖启动来应对流媒体推荐中随时间变化的用户偏好，具对数次线性遗憾界和强实证表现。

ABSTRACT

In real-world streaming recommender systems, user preferences evolve dynamically over time. Existing bandit-based methods treat time merely as a timestamp, neglecting its explicit relationship with user preferences and leading to suboptimal performance. Moreover, online learning methods often suffer from inefficient exploration-exploitation during the early online phase. To address these issues, we propose HyperBandit+, a novel contextual bandit policy that integrates a time-aware hypernetwork to adapt to time-varying user preferences and employs a large language model-assisted warm-start mechanism (LLM Start) to enhance exploration-exploitation efficiency in the early online phase. Specifically, HyperBandit+ leverages a neural network that takes time features as input and generates parameters for estimating time-varying rewards by capturing the correlation between time and user preferences. Additionally, the LLM Start mechanism employs multi-step data augmentation to simulate realistic interaction data for effective offline learning, providing warm-start parameters for the bandit policy in the early online phase. To meet real-time streaming recommendation demands, we adopt low-rank factorization to reduce hypernetwork training complexity. Theoretically, we rigorously establish a sublinear regret upper bound that accounts for both the hypernetwork and the LLM warm-start mechanism. Extensive experiments on real-world datasets demonstrate that HyperBandit+ consistently outperforms state-of-the-art baselines in terms of accumulated rewards.

研究动机与目标

• 在流媒体推荐中激发对随时间变化的用户偏好的处理。
• 通过时间变化的用户偏好矩阵对时间段相关的奖励进行建模。
• 通过基于LLM的离线暖启动提升寒启动阶段的探索–利用。
• 用低秩超网络参数化降低训练复杂度。
• 为所提出的框架提供理论遗憾界。

提出的方法

• 引入 HyperBandit+，它使用时间段感知的超网络来生成用户偏好映射。
• 将真实的时变奖励定义为 r*(u,a,p) = c_a^T Theta_p^* c_u，并随时间段学习 Theta_p。
• 采用 LLM Start 和 LLM-Enhanced Embedding 来分别增强潜在特征与观测特征。
• 使用低秩分解来压缩超网络输出并加速在线训练。
• 离线阶段包括 Euler Embedding 和 LLM-Enhanced Embedding 以获取侧信息；LLM Start 模拟数据用于暖启动。
• 通过带有潜在特征的闭式解的岭回归在线更新策略。

实验结果

研究问题

• RQ1 如何在流式上下文情境带宽中显式建模并适应周期性/随时间变化的用户偏好？
• RQ2 超网络是否可以生成时间段特定的用户偏好矩阵以提升推荐准确性？
• RQ3 基于LLM的数据增强是否提供有意义的暖启动以缓解早期阶段的探索–利用问题？
• RQ4 包含时间感知超网络和LLM基暖启动的带宽策略有哪些理论遗憾界？

主要发现

• HyperBandit+ 实现了考虑超网络和LLM暖启动机制的次线性遗憾界。
• 在真实数据集上的实证结果显示 HyperBandit+ 在累计奖励方面持续优于最先进基线。
• 低秩分解在不牺牲性能的前提下降低了训练复杂度。
• LLM Start 提供了有效的离线数据增强，改善早期在线的探索–利用效率。
• LLM-Enhanced Embedding 通过丰富用户和商品表征提升了侧信息质量。