[Paper Review] The Unfairness of Popularity Bias in Recommendation

Himan Abdollahpouri, Masoud Mansoury+2 more|arXiv (Cornell University)|Jul 31, 2019

Recommender Systems and TechniquesComputer Science19 references111 citations

TL;DR

The paper examines popularity bias from the user's perspective, defining user groups by interest in popular items and showing that many algorithms over-rely on popular items, especially harming niche users.

ABSTRACT

Recommender systems are known to suffer from the popularity bias problem: popular (i.e. frequently rated) items get a lot of exposure while less popular ones are under-represented in the recommendations. Research in this area has been mainly focusing on finding ways to tackle this issue by increasing the number of recommended long-tail items or otherwise the overall catalog coverage. In this paper, however, we look at this problem from the users' perspective: we want to see how popularity bias causes the recommendations to deviate from what the user expects to get from the recommender system. We define three different groups of users according to their interest in popular items (Niche, Diverse and Blockbuster-focused) and show the impact of popularity bias on the users in each group. Our experimental results on a movie dataset show that in many recommendation algorithms the recommendations the users get are extremely concentrated on popular items even if a user is interested in long-tail and non-popular items showing an extreme bias disparity.

Motivation & Objective

• Assess how popularity bias in data and algorithms causes recommendations to diverge from user expectations.
• Identify user groups with different interest in popular items.
• Evaluate how various recommendation algorithms propagate popularity bias for these groups.
• Highlight which algorithms better preserve the expected ratio of popular to non-popular items.

Proposed method

• Use MovieLens 1M dataset to analyze user profiles by the ratio of popular items in their ratings.
• Define three user groups: Niche (N), Diverse (D), and Blockbuster-focused (B) based on popular-item ratio.
• Compare multiple algorithms (User KNN, Item KNN, SVD++, Biased Matrix Factorization, plus Most Popular and Random) with similar precision.
• Measure the ratio of popular items in recommendations versus user profiles.
• Introduce GAP (Group Average Popularity) and ΔGAP as metrics to quantify shifts in item popularity exposure.

Experimental results

Research questions

• RQ1 RQ1: How much are different individuals or groups of users interested in popular items?
• RQ2 RQ2: How do algorithms’ popularity biases impact users with different degrees of interest in popular items.

Key findings

• Popularity bias is pervasive across tested algorithms, leading to recommendations that are far more popular-item-heavy than users’ profiles suggest.
• Niche users exhibit the largest discrepancy between their expected and actual ratio of popular items in recommendations.
• SVD++ shows comparatively better alignment with users’ expected popularity ratios than some other algorithms.
• Most Popular and Item KNN induce stronger correlations between item popularity and recommendations, amplifying bias.
• Group Average Popularity (GAP) and ΔGAP metrics reveal pronounced bias for niche users across algorithms.
• Overall, users with lower interest in popular items are disproportionately underserved by the studied recommendation algorithms.