[Paper Review] Individual Fairness in Pipelines

Cynthia Dwork, Christina Ilvento+1 more|arXiv (Cornell University)|Jan 1, 2020

Ethics and Social Impacts of AISocial Sciences23 references11 citations

TL;DR

This paper introduces a formal framework for ensuring individual fairness in pipeline systems—sequential workflows where individuals are filtered or selected in stages, and later decisions depend on cohort composition. It shows that naively fair components can lead to arbitrarily unfair outcomes due to cohort-dependent scoring, and proposes robust cohort selection mechanisms that remain fair under a wide range of future scoring policies by enforcing similarity in cohort context for similar individuals.

ABSTRACT

It is well understood that a system built from individually fair components may not itself be individually fair. In this work, we investigate individual fairness under pipeline composition. Pipelines differ from ordinary sequential or repeated composition in that individuals may drop out at any stage, and classification in subsequent stages may depend on the remaining "cohort" of individuals. As an example, a company might hire a team for a new project and at a later point promote the highest performer on the team. Unlike other repeated classification settings, where the degree of unfairness degrades gracefully over multiple fair steps, the degree of unfairness in pipelines can be arbitrary, even in a pipeline with just two stages. Guided by a panoply of real-world examples, we provide a rigorous framework for evaluating different types of fairness guarantees for pipelines. We show that naïve auditing is unable to uncover systematic unfairness and that, in order to ensure fairness, some form of dependence must exist between the design of algorithms at different stages in the pipeline. Finally, we provide constructions that permit flexibility at later stages, meaning that there is no need to lock in the entire pipeline at the time that the early stage is constructed.

Motivation & Objective

• To address the critical gap in algorithmic fairness where individually fair components in pipelines can still produce unfair outcomes due to cohort-dependent decision-making.
• To formalize fairness guarantees for pipeline systems that differ from standard sequential composition due to cohort filtering and context-sensitive scoring.
• To demonstrate that naive auditing fails to detect systemic unfairness in pipelines, especially when cohort selection exploits future scoring policies.
• To construct cohort selection mechanisms that are robust to a broad class of future scoring functions, even when those functions are chosen after the initial selection.

Proposed method

• Proposes a formal model of cohort pipelines with individual fairness constraints, distinguishing between unconditional and conditional fairness metrics.
• Introduces conditional robustness as a fairness guarantee, ensuring that the final outcome distribution remains fair regardless of the downstream scoring policy.
• Defines a 'quality-symmetric' cohort selection mechanism that assigns similar individuals to similar distributions over permissible cohorts.
• Uses Lipschitz continuity with respect to a metric over outcome distributions to formalize fairness under both expected score and mass-moving distance (MMD) metrics.
• Establishes that conditional robustness implies unconditional robustness up to a Lipschitz constant, enabling transferability of fairness guarantees.
• Provides constructions that permit flexibility in later-stage scoring by encoding fairness constraints into early-stage cohort design.

Experimental results

Research questions

• RQ1 Can individually fair components in a pipeline still result in unfair overall outcomes due to cohort-dependent scoring?
• RQ2 Why does standard auditing fail to detect unfairness in pipeline systems, even when individual components are fair?
• RQ3 What structural conditions on cohort selection are necessary and sufficient to ensure fairness across a range of downstream scoring policies?
• RQ4 How can a cohort selection mechanism be designed to be robust to future, unknown scoring functions while preserving fairness?
• RQ5 What mathematical frameworks (e.g., MMD, expected score) best capture fairness in pipeline compositions?

Key findings

• Naive auditing of fairness in pipelines fails to detect systematic unfairness, as malicious pipelines can be constructed that appear fair at each stage but produce biased outcomes.
• The degree of unfairness in pipelines can be arbitrary—even with just two stages—contrary to the graceful degradation seen in standard sequential composition.
• Robustness to downstream scoring policies requires that similar individuals be assigned to similar distributions over cohort contexts, not just similar outcomes.
• Conditional robustness (fairness given a cohort) implies unconditional robustness (fairness overall) up to a Lipschitz constant of (α + 1), enabling transfer of fairness guarantees.
• Mass-moving distance (MMD) is at least as strong as expected score in measuring fairness, with |E[X1]−E[X2]| ≤3MMD(X1,X2), providing a tighter bound for fairness evaluation.
• The paper constructs cohort selection mechanisms that are robust to a rich family of scoring policies using a simple policy description, enabling flexibility in later-stage decisions without compromising fairness.