[Paper Review] Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production Systems

Chloé Bakalar, Renata da Costa Barreto+14 more|arXiv (Cornell University)|Mar 10, 2021

Ethics and Social Impacts of AI56 references10 citations

TL;DR

This paper presents a practical framework for integrating algorithmic fairness into large-scale production systems by disentangling normative policy decisions from empirical implementation metrics. It introduces a measurement approach grounded in explicit trade-offs of harms and benefits across groups, using threshold analysis for ML models and Signal Detection Theory to infer human labelers' decision thresholds, enabling actionable fairness interventions in real-world systems.

ABSTRACT

Many technical approaches have been proposed for ensuring that decisions made by machine learning systems are fair, but few of these proposals have been stress-tested in real-world systems. This paper presents an example of one team's approach to the challenge of applying algorithmic fairness approaches to complex production systems within the context of a large technology company. We discuss how we disentangle normative questions of product and policy design (like, "how should the system trade off between different stakeholders' interests and needs?") from empirical questions of system implementation (like, "is the system achieving the desired tradeoff in practice?"). We also present an approach for answering questions of the latter sort, which allows us to measure how machine learning systems and human labelers are making these tradeoffs across different relevant groups. We hope our experience integrating fairness tools and approaches into large-scale and complex production systems will be useful to other practitioners facing similar challenges, and illuminating to academics and researchers looking to better address the needs of practitioners.

Motivation & Objective

• Address the gap between theoretical fairness methods and real-world deployment in large-scale systems.
• Disentangle normative policy questions (e.g., who bears costs) from empirical implementation questions (e.g., does the system achieve the intended trade-off?).
• Develop a measurement framework that makes the costs and benefits of decisions explicit across different demographic groups.
• Apply fairness assessment to both ML-driven decisions and human labeler behavior in production environments.
• Provide a scalable, context-aware approach to fairness that supports actionable insights for practitioners and researchers.

Proposed method

• Disentangle fairness into three levels: product design, policy formulation, and system implementation to guide targeted interventions.
• Use threshold prevalence analysis to measure how ML models prioritize false positives and false negatives across groups.
• Apply Signal Detection Theory (SDT) to infer latent decision thresholds used by human labelers, treating them as psychological decision-makers.
• Frame fairness as a trade-off between harms (false positives, false negatives) and benefits, making value-laden choices explicit.
• Measure fairness using metrics grounded in real-world impact rather than abstract statistical parity, ensuring alignment with stakeholder priorities.
• Conduct root-cause analysis of fairness issues by diagnosing model behavior and human labeling practices, especially in cases of label bias.

Experimental results

Research questions

• RQ1 How can normative fairness decisions (e.g., trade-offs between stakeholders) be separated from empirical fairness measurement in production systems?
• RQ2 What metrics can effectively measure implementation-level fairness in ML systems and human labeling processes?
• RQ3 How can the implicit value-laden choices in fairness metrics be made explicit and auditable?
• RQ4 To what extent can Signal Detection Theory be used to infer and correct for human labeler bias in real-world systems?
• RQ5 What are the limitations of standard fairness metrics when applied to systems with measurement error, label inaccuracies, or adversarial behavior?

Key findings

• The approach successfully disentangles normative decisions (e.g., who should bear the cost of errors) from empirical measurements, enabling clearer stakeholder alignment.
• Threshold prevalence analysis revealed systematic differences in false positive and false negative rates across demographic groups in ML systems, indicating biased prioritization.
• Signal Detection Theory enabled the inference of latent decision thresholds used by human labelers, exposing inconsistencies and biases in labeling behavior.
• Labeling bias was found to be highly sensitive to guidelines, selection processes, and incentives, suggesting that behavioral interventions can correct imbalances.
• Measurement errors in sensitive attribute labeling—especially when inferred rather than self-reported—can significantly distort fairness assessments.
• No single metric suffices for complex systems; fairness must be evaluated at component and system-wide levels due to emergent interactions in feedback loops.