[Paper Review] On the (im)possibility of fairness

Sorelle A. Friedler, Carlos Scheidegger+1 more|arXiv (Cornell University)|Sep 23, 2016

Ethics and Social Impacts of AISocial Sciences22 references263 citations

TL;DR

The paper develops a space-based formal framework for algorithmic fairness, introducing construct, observed, and decision spaces, and shows that fairness guarantees depend on strong worldviews about the construct–observed relationship (WYSIWYG vs. WAE). It also defines notions of structural bias and discrimination and analyzes when fair or non-discriminatory mechanisms can be guaranteed.

ABSTRACT

What does it mean for an algorithm to be fair? Different papers use different notions of algorithmic fairness, and although these appear internally consistent, they also seem mutually incompatible. We present a mathematical setting in which the distinctions in previous papers can be made formal. In addition to characterizing the spaces of inputs (the "observed" space) and outputs (the "decision" space), we introduce the notion of a construct space: a space that captures unobservable, but meaningful variables for the prediction. We show that in order to prove desirable properties of the entire decision-making process, different mechanisms for fairness require different assumptions about the nature of the mapping from construct space to decision space. The results in this paper imply that future treatments of algorithmic fairness should more explicitly state assumptions about the relationship between constructs and observations.

Motivation & Objective

• Introduce a mathematical theory of fairness as transformations between construct, observed, and decision spaces.
• Characterize how spaces interact and how distortions affect fairness and discrimination.
• Prove that fairness guarantees require strong assumptions about the construct–observed mapping.
• Differentiate worldviews (WYSIWYG vs. WAE) and link them to fairness notions and mechanisms.
• Define and quantify structural bias and direct/non-discrimination in a rigorous space-based setting.

Proposed method

• Define the construct space, observed space, and decision space with associated metrics.
• Introduce distortions, coupling measures, Wasserstein distance, and Gromov-Wasserstein distance to compare spaces.
• Formally define fairness as closeness preservation between construct and decision spaces.
• Present worldviews: WYSIWYG (construct≈observed) and WAE (we’re all equal) and their implications.
• Define mechanisms: individual fairness mechanisms (IFM) and group fairness mechanisms (GFM).
• Prove that under WYSIWYG, an IFM with disturbances yields fairness guarantees; discuss limitations under weaker assumptions.

Experimental results

Research questions

• RQ1 How can fairness be formalized when decision-making maps between construct, observed, and decision spaces?
• RQ2 Under what assumptions can fairness guarantees be achieved or fail in this tripartite space framework?
• RQ3 How do structural bias and group differences affect the mapping between spaces and the resulting fairness outcomes?
• RQ4 How do different worldviews (WYSIWYG vs. WAE) influence the design of fair or non-discriminatory mechanisms?

Key findings

• Fairness guarantees depend on strong assumptions about the relation between construct and observed spaces.
• A WYSIWYG worldview can yield fairness guarantees for suitable mechanisms under small distortion.
• Structural bias can be quantified via group skew between spaces and affects non-discrimination assessments.
• There is a fundamental distinction between individual fairness mechanisms and group fairness mechanisms in this space framework.
• Most fairness methods implicitly assume particular space relationships, which the framework makes explicit.