[Paper Review] Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction

Nina Grgić-Hlača, Elissa M. Redmiles+2 more|arXiv (Cornell University)|Feb 26, 2018

Ethics and Social Impacts of AI40 references71 citations

TL;DR

The paper empirically studies how people perceive fairness in using features of the COMPAS criminal risk tool, proposing an eight-property framework that predicts fairness judgments from latent feature properties with high accuracy. It reveals multi-dimensional unfairness concerns beyond discrimination and substantial disagreement among respondents.

ABSTRACT

As algorithms are increasingly used to make important decisions that affect human lives, ranging from social benefit assignment to predicting risk of criminal recidivism, concerns have been raised about the fairness of algorithmic decision making. Most prior works on algorithmic fairness normatively prescribe how fair decisions ought to be made. In contrast, here, we descriptively survey users for how they perceive and reason about fairness in algorithmic decision making. A key contribution of this work is the framework we propose to understand why people perceive certain features as fair or unfair to be used in algorithms. Our framework identifies eight properties of features, such as relevance, volitionality and reliability, as latent considerations that inform people's moral judgments about the fairness of feature use in decision-making algorithms. We validate our framework through a series of scenario-based surveys with 576 people. We find that, based on a person's assessment of the eight latent properties of a feature in our exemplar scenario, we can accurately (> 85%) predict if the person will judge the use of the feature as fair. Our findings have important implications. At a high-level, we show that people's unfairness concerns are multi-dimensional and argue that future studies need to address unfairness concerns beyond discrimination. At a low-level, we find considerable disagreements in people's fairness judgments. We identify root causes of the disagreements, and note possible pathways to resolve them.

Motivation & Objective

• Identify how people judge the fairness of using individual features in algorithmic decisions.
• Propose a framework of latent feature properties that influence fairness judgments.
• Empirically validate the framework with large-scale surveys (n=576) in a criminal risk prediction context.
• Analyze consensus and disagreements in fairness judgments and their causes.
• Discuss implications for designing fairer algorithmic decision-making systems.

Proposed method

• Introduce an eight-property latent framework (reliability, relevance, privacy, volitionality, causes outcome, causes vicious cycle, causes disparity in outcomes, caused by sensitive group membership).
• Design scenario-based surveys around COMPAS inputs to collect fairness judgments from 576 participants.
• Conduct pilot and main surveys to assess both latent properties and fairness judgments.
• Use two datasets (AMT and SSI) to assess generalizability and consensus.
• Apply logistic regression with L2 regularization to predict fairness judgments from latent-property assessments.

Experimental results

Research questions

• RQ1 What latent properties do people implicitly use when judging whether a feature is fair to use in a decision-making scenario?
• RQ2 Can fairness judgments be predicted from assessments of these latent properties?
• RQ3 How do consensus and disagreements arise in fairness judgments across different features and populations?
• RQ4 Do unfairness concerns extend beyond discrimination to other feature characteristics?
• RQ5 What implications do these findings have for designing fair algorithmic decision-making systems?

Key findings

• A majority of respondents judged about half of the COMPAS features as unfair to use for bail decisions.
• Eight latent properties sufficiently explain fairness judgments, with six shown as statistically significant predictors in their analysis.
• Fairness judgments can be predicted with high accuracy (> 85%) from latent-property assessments using a simple classifier.
• Most fairness considerations identified were unrelated to discrimination, highlighting additional unfairness concerns.
• There is considerable disagreement in fairness judgments for several features, driven largely by differing assessments of latent properties, particularly causal ones.
• A simple classifier trained on latent properties accurately predicts fairness judgments, suggesting latent properties could be objectively determined while moral reasoning could be elicited from surveys.