[Paper Review] The Measure and Mismeasure of Fairness

Sam Corbett‐Davies, Gaebler, Johann D.+3 more|arXiv (Cornell University)|Jul 31, 2018

Ethics and Social Impacts of AISocial Sciences50 references472 citations

TL;DR

The paper categorizes popular formal fairness definitions into two families, shows they often yield Pareto-dominated policies, and argues for a consequentialist, policy-aligned approach to equitable algorithm design.

ABSTRACT

The field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last decade, several formal, mathematical definitions of fairness have gained prominence. Here we first assemble and categorize these definitions into two broad families: (1) those that constrain the effects of decisions on disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions typically result in strongly Pareto dominated decision policies. For example, in the case of college admissions, adhering to popular formal conceptions of fairness would simultaneously result in lower student-body diversity and a less academically prepared class, relative to what one could achieve by explicitly tailoring admissions policies to achieve desired outcomes. In this sense, requiring that these fairness definitions hold can, perversely, harm the very groups they were designed to protect. In contrast to axiomatic notions of fairness, we argue that the equitable design of algorithms requires grappling with their context-specific consequences, akin to the equitable design of policy. We conclude by listing several open challenges in fair machine learning and offering strategies to ensure algorithms are better aligned with policy goals.

Motivation & Objective

• Categorize existing formal fairness definitions into two families based on whether they constrain decision disparities or constrain protected attributes’ influence on decisions.
• Demonstrate analytically and empirically that these fairness notions often lead to strongly Pareto dominated policies across utility functions.
• Illustrate how conventional fairness criteria can harm the very groups they aim to protect in real-world settings like college admissions.
• Advise on designing equitable algorithms by foregrounding policy goals and context-specific trade-offs.

Proposed method

• Propose a formal setting with individuals characterized by covariates X, protected attributes A, binary decision D, and outcome Y under a budget constraint b.
• Define and discuss multiple fairness notions, including demographic parity, equalized false positive rates, and causal/counterfactual fairness variants.
• present path-specific fairness using causal DAGs to separate legitimate vs illegitimate paths from protected attributes to decisions.
• Provide counterfactual and path-specific constructs to analyze how changing protected attributes would affect decisions along specified causal paths.
• Argue for a consequentialist framework that treats algorithmic decisions as policy instruments and examines their broader impacts.

Experimental results

Research questions

• RQ1 What are the main families of formal fairness definitions used in fair ML, and how do they differ in their treatment of disparities vs. protected attributes?
• RQ2 Do commonly enforced fairness criteria lead to Pareto-dominated decision policies across natural utility frames?
• RQ3 How can fairness be conceptualized in a causal/path-specific sense, and what are the practical implications for policy-aligned algorithm design?
• RQ4 What guidance can be offered for designing equitable algorithms that align with policy goals rather than axiomatic fairness through unawareness?
• RQ5 What concrete recommendations emerge for implementing equitable decisions in settings with and without externalities (e.g., college admissions vs. medical screening)?

Key findings

• Popular fairness definitions can yield strongly Pareto dominated policies across a range of utility functions.
• Enforcing axiomatic fairness criteria can reduce diversity and academic preparedness in college admissions relative to tailored, outcome-driven policies.
• Blinding protected attributes does not universally achieve calibration and can obscure meaningful disparities.
• Path-specific and counterfactual fairness offer a nuanced view by isolating legitimate causal paths from illegitimate ones in decision making.
• A consequentialist, policy-aware approach to algorithm design better aligns with real-world equity goals and trade-offs.