[Paper Review] Equality of Opportunity in Supervised Learning

Moritz Hardt, Eric Price+1 more|arXiv (Cornell University)|Oct 7, 2016

Ethics and Social Impacts of AI6 references1,909 citations

TL;DR

The paper defines an oblivious fairness criterion (equalized odds and equal opportunity) for predicting Y from X with protected attribute A, and shows how to post-process any predictor to remove discrimination while preserving or improving utility.

ABSTRACT

We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. Assuming data about the predictor, target, and membership in the protected group are available, we show how to optimally adjust any learned predictor so as to remove discrimination according to our definition. Our framework also improves incentives by shifting the cost of poor classification from disadvantaged groups to the decision maker, who can respond by improving the classification accuracy. In line with other studies, our notion is oblivious: it depends only on the joint statistics of the predictor, the target and the protected attribute, but not on interpretation of individualfeatures. We study the inherent limits of defining and identifying biases based on such oblivious measures, outlining what can and cannot be inferred from different oblivious tests. We illustrate our notion using a case study of FICO credit scores.

Motivation & Objective

• Formalize an oblivious notion of non-discrimination with respect to a protected attribute in supervised learning.
• Propose a post-processing framework to derive non-discriminating predictors from any learned predictor or score.
• Demonstrate that Bayes-optimal non-discriminating predictors can be obtained from Bayes-optimal regressors.
• Analyze inherent limitations of oblivious fairness approaches and the identifiability of discrimination.

Proposed method

• Define equalized odds and equal opportunity in terms of conditional independence given the target Y.
• Introduce derived predictors that depend only on the initial predictor and the protected attribute A.
• Formulate the problem as a linear program (for binary Y and A) to find an equalized odds predictor with minimal loss.
• Extend the approach to real-valued scores R by using factorized thresholds and randomized thresholding to achieve equalized odds.
• Show that a Bayes-optimal equalized odds predictor can be derived from the Bayes-optimal regressor, and provide near-optimality results using conditional Kolmogorov distance.

Experimental results

Research questions

• RQ1 How can one define and measure discrimination in a way that is oblivious to individual features and depends only on joint statistics (Y, Ŷ, A)?
• RQ2 How can any learned predictor be post-processed to satisfy equalized odds or equal opportunity with minimal loss of predictive accuracy?
• RQ3 What is the relationship between Bayes-optimal predictors and bias-robust versions under oblivious fairness constraints?
• RQ4 What are the limitations of oblivious tests in identifying the true underlying discrimination structures?

Key findings

• An easily checkable, interpretable notion of avoiding discrimination based on protected attributes is proposed and is shown to align with improved utility.
• A post-processing step can derive a non-discriminating predictor from any binary predictor or score without retraining the original model.
• For binary Y and A, the derived equalized odds predictor can be obtained via a linear program with coefficients computed from the joint distribution of (Ŷ, A, Y).
• Equal opportunity can be achieved via a similar approach with a weaker constraint, and thresholds can be optimized efficiently (e.g., via ternary search).
• A Bayes-optimal equalized odds predictor is derivable from the Bayes-optimal regressor, and near-optimality results quantify the loss when the regressor deviates from Bayes optimality.