[Paper Review] Managing Popularity Bias in Recommender Systems with Personalized Re-ranking

Himan Abdollahpouri, Robin Burke+1 more|arXiv (Cornell University)|Jan 22, 2019

Recommender Systems and TechniquesComputer Science22 references83 citations

TL;DR

The paper adapts xQuAD-style diversification into a personalized re-ranking framework to reduce popularity bias by promoting long-tail items after output from any recommender, comparing Binary and Smooth variants against a regularization baseline.

ABSTRACT

Many recommender systems suffer from popularity bias: popular items are recommended frequently while less popular, niche products, are recommended rarely or not at all. However, recommending the ignored products in the `long tail' is critical for businesses as they are less likely to be discovered. In this paper, we introduce a personalized diversification re-ranking approach to increase the representation of less popular items in recommendations while maintaining acceptable recommendation accuracy. Our approach is a post-processing step that can be applied to the output of any recommender system. We show that our approach is capable of managing popularity bias more effectively, compared with an existing method based on regularization. We also examine both new and existing metrics to measure the coverage of long-tail items in the recommendation.

Motivation & Objective

• Motivate the problem of popularity bias and the importance of long-tail item representation in recommender systems.
• Propose a post-processing re-ranking approach that adjusts an existing recommendation list to improve long-tail coverage while preserving accuracy.
• Enable personalization by weighting long-tail promotion according to individual user interest in long-tail items.
• Compare re-ranking approaches with a regularization-based baseline across datasets to assess long-tail coverage and ranking quality.

Proposed method

• Adapt the xQuAD result diversification framework to control popularity bias in recommender outputs.
• Formulate a re-ranking score that combines base predicted relevance with a personalized diversification term across short-head and long-tail categories (Gamma and Gamma′).
• Introduce two variants for estimating the diversification term: Binary xQuAD (category coverage indicator) and Smooth xQuAD (ratio-based coverage).
• Compute P(v|u) from the base recommender and P(d|u) for user preference over short-head vs long-tail, using Equation 4 to balance accuracy and diversity.
• Apply the re-ranking iteratively to generate a final list S of desired length from an initial ranked list R, with a configurable lambda controlling the bias-accuracy trade-off.
• Evaluate using long-tail coverage metrics (APLT, ACLT) and accuracy metric (NDCG), plus ARP for exposure.
• Datasets used include MovieLens 1M and Epinions, with items split into short-head and long-tail based on rating distribution (top ~80% vs bottom ~20%).

Experimental results

Research questions

• RQ1 Can personalized re-ranking reduce popularity bias while preserving recommendation accuracy?
• RQ2 How do Binary and Smooth xQuAD variants compare to a regularized long-tail diversification baseline in terms of long-tail coverage and rank quality?
• RQ3 What is the impact of dataset sparsity (MovieLens vs. Epinions) on long-tail promotion and diversity-accuracy trade-offs?
• RQ4 Is ARP alone an adequate measure for evaluating long-tail promotion, or should it be used with additional metrics like APLT and ACLT?
• RQ5 Can the approach be extended to multi-stakeholder settings in recommender systems?

Key findings

• Re-ranking methods improve long-tail exposure compared to a regularization baseline, with Smooth xQuAD often delivering stronger long-tail coverage.
• Binary xQuAD tends to adjust lists minimally, while Smooth xQuAD balances increasing long-tail items with maintaining ranking quality.
• LT-Reg shows less effectiveness in promoting long-tail items across evaluated metrics (APLT and ACLT) in the reported experiments.
• On both datasets, ARP alone does not fully capture long-tail diversity, and should be evaluated alongside APLT and ACLT to assess effectiveness.
• The approach demonstrates that re-ranking can achieve better long-tail representation with only moderate loss in NDCG, highlighting a favorable diversity-accuracy trade-off.
• Results indicate the Epinions (more sparse) dataset presents a greater challenge for long-tail diversification than MovieLens, with different relative performances for Binary and Smooth variants.