[论文解读] Unbiased Cascade Bandits: Mitigating Exposure Bias in Online Learning to Rank Recommendation

Masoud Mansoury, Himan Abdollahpouri+4 more|arXiv (Cornell University)|Aug 7, 2021

Advanced Bandit Algorithms Research参考文献 31被引用 23

一句话总结

本文提出Unbiased Cascade Bandits，一种将折扣机制整合进线性级联Bandit算法的模型，以缓解在线排序学习推荐系统中的曝光偏差。通过动态降低频繁曝光物品的效用，该方法在累积奖励损失最小的前提下，显著提升了物品与供应商的曝光公平性，该结论已在两个真实世界数据集上使用三种Bandit算法得到验证。

ABSTRACT

Exposure bias is a well-known issue in recommender systems where items and suppliers are not equally represented in the recommendation results. This is especially problematic when bias is amplified over time as a few popular items are repeatedly over-represented in recommendation lists. This phenomenon can be viewed as a recommendation feedback loop: the system repeatedly recommends certain items at different time points and interactions of users with those items will amplify bias towards those items over time. This issue has been extensively studied in the literature on model-based or neighborhood-based recommendation algorithms, but less work has been done on online recommendation models such as those based on multi-armed Bandit algorithms. In this paper, we study exposure bias in a class of well-known bandit algorithms known as Linear Cascade Bandits. We analyze these algorithms on their ability to handle exposure bias and provide a fair representation for items and suppliers in the recommendation results. Our analysis reveals that these algorithms fail to treat items and suppliers fairly and do not sufficiently explore the item space for each user. To mitigate this bias, we propose a discounting factor and incorporate it into these algorithms that controls the exposure of items at each time step. To show the effectiveness of the proposed discounting factor on mitigating exposure bias, we perform experiments on two datasets using three cascading bandit algorithms and our experimental results show that the proposed method improves the exposure fairness for items and suppliers.

研究动机与目标

• 探究级联Bandit算法是否在在线学习排序推荐系统中天然缓解曝光偏差。
• 分析现有级联Bandit算法随时间推移在全物品空间中探索的公平程度。
• 通过基于历史曝光的动态折扣机制，解决这些算法中持续存在的曝光偏差问题。
• 评估所提方法在不牺牲推荐相关性的情况下，提升物品与供应商曝光公平性的有效性。

提出的方法

• 引入一种新颖的折扣因子，根据物品在先前时间步的累积曝光量降低其效用。
• 通过引入基于曝光的折扣因子，修改级联Bandit算法的效用函数，以鼓励对曝光不足物品的探索。
• 将该方法应用于三种级联Bandit算法：CascadeLSB、CascadeLinUCB和CascadeHybrid，增强其探索行为。
• 使用超参数 $ c $ 控制折扣效应的强度，最优值通过实验在 $ c = 0.5 $ 和 $ c = 1 $ 处确定。
• 采用n步遗憾和物品覆盖率（IC）作为主要指标，评估性能与公平性之间的权衡。

实验结果

研究问题

• RQ1 现有级联Bandit算法在多大程度上能随时间公平地探索并曝光所有物品与供应商？
• RQ2 基于曝光的动态折扣机制是否能在不降低推荐性能的前提下提升物品与供应商曝光的公平性？
• RQ3 折扣超参数 $ c $ 的选择如何影响遗憾与曝光公平性之间的权衡？
• RQ4 所提方法是否在曝光公平性方面优于基线级联Bandit算法，同时保持较高的累积奖励？

主要发现

• 与原始算法相比，所提出的Unbiased Cascade Bandits在物品覆盖率（IC）方面有显著提升，在MovieLens数据集上，当 $ c = 1 $ 时，IC最高达到98%。
• 在Last.fm数据集上，UnbiasedCascadeLSB在 $ c = 0.5 $ 时的物品覆盖率比原始版本高出6.3%，且n步遗憾几乎未增加。
• 对于 $ c ∈ \{0.5, 1\} $，所提方法在两个数据集上均一致优于原始算法，在物品覆盖率和公平性指标上表现更优。
• 仅调整 $ c $ 无法提升公平性，表明折扣机制本身是关键，而非超参数调优问题。
• 该方法保持了较高的奖励性能，表现为尽管曝光公平性显著提升，n步遗憾的增加却微乎其微。