[Paper Review] Factors Influencing Perceived Fairness in Algorithmic Decision-Making: Algorithm Outcomes, Development Procedures, and Individual Differences

Ruotong Wang, F. Maxwell Harper+1 more|arXiv (Cornell University)|Jan 27, 2020

Ethics and Social Impacts of AISocial Sciences68 references30 citations

TL;DR

This CHI 2020 study uses a randomized online MTurk experiment to show that favorable individual outcomes and unbiased group treatments increase perceived fairness, with outcome effects often exceeding bias effects and with education and development procedures moderating these perceptions.

ABSTRACT

Algorithmic decision-making systems are increasingly used throughout the public and private sectors to make important decisions or assist humans in making these decisions with real social consequences. While there has been substantial research in recent years to build fair decision-making algorithms, there has been less research seeking to understand the factors that affect people's perceptions of fairness in these systems, which we argue is also important for their broader acceptance. In this research, we conduct an online experiment to better understand perceptions of fairness, focusing on three sets of factors: algorithm outcomes, algorithm development and deployment procedures, and individual differences. We find that people rate the algorithm as more fair when the algorithm predicts in their favor, even surpassing the negative effects of describing algorithms that are very biased against particular demographic groups. We find that this effect is moderated by several variables, including participants' education level, gender, and several aspects of the development procedure. Our findings suggest that systems that evaluate algorithmic fairness through users' feedback must consider the possibility of outcome favorability bias.

Motivation & Objective

• Understand how individual favorable outcomes influence perceived fairness of algorithmic decisions.
• Assess how group-based unbiased versus biased outcomes affect perceived fairness.
• Examine the impact of development procedures (transparency and human involvement) on perceived fairness.
• Investigate how individual differences (education, computer literacy, demographics) modulate fairness perceptions.

Proposed method

• Conduct a randomized online experiment on MTurk (N ~ 579 after exclusions).
• Manipulate six factors: (un)favorable outcome, (un)biased group treatment, transparency level, development/design team, model type (ML vs rules), and decision type (algorithm-only vs mixed).
• Present participants with a description of an MTurk Master qualification algorithm and the outcome (pass/fail) while varying outcome and procedure details.
• Measure perceived fairness with six 7-point Likert items (Cronbach’s alpha = 0.91).
• Analyze data using linear regression models predicting perceived fairness from outcomes, procedures, and individual differences (control: self-expectation).

Experimental results

Research questions

• RQ1 How do favorable individual outcomes and unbiased group treatment affect perceived algorithmic fairness?
• RQ2 What is the effect of development procedures (transparency and human involvement) on fairness perceptions?
• RQ3 Do individual differences (education, computer literacy, demographics) modulate perceived fairness?
• RQ4 Is the impact of outcome favorability larger than the impact of unbiased treatment on fairness judgments?
• RQ5 How do education and procedure transparency interact with fairness perceptions?

Key findings

• Perceived fairness rises with favorable outcome for the individual and with unbiased treatment across groups.
• The effect of a favorable individual outcome on fairness is larger than the effect of unbiased group treatment.
• Higher education attenuates the fairness boost from favorable outcomes and amplifies sensitivity to biased group treatments.
• Describing development with outsourcing or higher transparency can amplify the negative impact of group biases on perceived fairness.
• More human involvement in development generally increases perceived fairness, supporting a human-in-the-loop view.
• Overall, outcome favorability can bias fairness judgments beyond statistical fairness markers, indicating outcome favorability bias.