[Paper Review] Bias and Debias in Recommender System: A Survey and Future Directions

Jiawei Chen, Hande Dong+4 more|arXiv (Cornell University)|Oct 7, 2020

Recommender Systems and TechniquesComputer Science160 citations

TL;DR

A systematic survey of seven biases in recommender systems, their causal explanations, and a taxonomy of debiasing methods, highlighting open challenges and future directions.

ABSTRACT

While recent years have witnessed a rapid growth of research papers on recommender system (RS), most of the papers focus on inventing machine learning models to better fit user behavior data. However, user behavior data is observational rather than experimental. This makes various biases widely exist in the data, including but not limited to selection bias, position bias, exposure bias, and popularity bias. Blindly fitting the data without considering the inherent biases will result in many serious issues, e.g., the discrepancy between offline evaluation and online metrics, hurting user satisfaction and trust on the recommendation service, etc. To transform the large volume of research models into practical improvements, it is highly urgent to explore the impacts of the biases and perform debiasing when necessary. When reviewing the papers that consider biases in RS, we find that, to our surprise, the studies are rather fragmented and lack a systematic organization. The terminology ``bias'' is widely used in the literature, but its definition is usually vague and even inconsistent across papers. This motivates us to provide a systematic survey of existing work on RS biases. In this paper, we first summarize seven types of biases in recommendation, along with their definitions and characteristics. We then provide a taxonomy to position and organize the existing work on recommendation debiasing. Finally, we identify some open challenges and envision some future directions, with the hope of inspiring more research work on this important yet less investigated topic. The summary of debiasing methods reviewed in this survey can be found at \url{https://github.com/jiawei-chen/RecDebiasing}.

Motivation & Objective

• Provide a systematic overview of biases in recommender systems and their characteristics.
• Clarify inconsistent definitions of bias across RS literature using causality-based explanations.
• Survey existing debiasing methodologies and organize them into a coherent taxonomy.
• Identify open challenges and propose future directions to guide RS research and practice.

Proposed method

• Classify seven types of biases in RS and define their characteristics with causal graphs.
• Present a taxonomy of debiasing techniques and discuss strengths and weaknesses of each approach.
• Summarize the interaction between data, model, and results biases within the RS feedback loop.
• Highlight open challenges and potential future research directions.

Experimental results

Research questions

• RQ1 What are the seven identified biases in recommender systems and how are they defined and characterized?
• RQ2 How can debiasing methods be categorized, what are their relative strengths and weaknesses, and where do they apply within the RS lifecycle?
• RQ3 How do biases propagate and amplify through the RS feedback loop, and what are the open challenges for mitigating them?

Key findings

• Seven biases in RS are identified: selection bias, exposure bias, conformity bias, position bias, inductive bias, popularity bias, and unfairness.
• Biases are analyzed with causality-based explanations to show how they arise and affect data, models, and results.
• A taxonomy of debiasing techniques is provided, outlining methods such as propensity scores, data imputation, joint generative models, doubly robust models, and causality-based approaches.
• Biases can amplify through the feedback loop, leading to issues like echo chambers and reduced diversity, while some biases can be self-reinforcing in data.
• The survey synthesizes over 180 related papers and emphasizes open challenges and future directions for bias mitigation in RS.