[Paper Review] From Soft Classifiers to Hard Decisions: How fair can we be?

Ran Canetti, Aloni Cohen+4 more|arXiv (Cornell University)|Oct 3, 2018

Ethics and Social Impacts of AI4 references4 citations

TL;DR

This paper investigates post-processing calibrated soft classifiers to achieve group fairness in binary decision-making, showing that equalizing predictive values (PPV/NPV) via global thresholds is generally impossible. It proposes deferral-based post-processing that enables equalization of PPV, NPV, FPR, and FNR across groups, circumventing known impossibility results under standard thresholding.

ABSTRACT

A popular methodology for building binary decision-making classifiers in the presence of imperfect information is to first construct a non-binary "scoring" classifier that is calibrated over all protected groups, and then to post-process this score to obtain a binary decision. We study the feasibility of achieving various fairness properties by post-processing calibrated scores, and then show that deferring post-processors allow for more fairness conditions to hold on the final decision. Specifically, we show: 1. There does not exist a general way to post-process a calibrated classifier to equalize protected groups' positive or negative predictive value (PPV or NPV). For certain "nice" calibrated classifiers, either PPV or NPV can be equalized when the post-processor uses different thresholds across protected groups, though there exist distributions of calibrated scores for which the two measures cannot be both equalized. When the post-processing consists of a single global threshold across all groups, natural fairness properties, such as equalizing PPV in a nontrivial way, do not hold even for "nice" classifiers. 2. When the post-processing is allowed to `defer' on some decisions (that is, to avoid making a decision by handing off some examples to a separate process), then for the non-deferred decisions, the resulting classifier can be made to equalize PPV, NPV, false positive rate (FPR) and false negative rate (FNR) across the protected groups. This suggests a way to partially evade the impossibility results of Chouldechova and Kleinberg et al., which preclude equalizing all of these measures simultaneously. We also present different deferring strategies and show how they affect the fairness properties of the overall system. We evaluate our post-processing techniques using the COMPAS data set from 2016.

Motivation & Objective

• To analyze the limits of post-processing calibrated soft classifiers to achieve group fairness in binary decision-making.
• To investigate whether fairness properties like equalized PPV and NPV can be achieved through thresholding across protected groups.
• To explore how deferral mechanisms can enable equalization of multiple fairness metrics (PPV, NPV, FPR, FNR) simultaneously.
• To evaluate the proposed methods on real-world data (COMPAS) and assess trade-offs in fairness and deferral rates.
• To examine the technical and procedural implications of deferral models in fairness-aware decision systems.

Proposed method

• Post-process calibrated soft classifiers using group-aware thresholds to adjust decision boundaries per protected group.
• Introduce a deferral mechanism where some inputs are passed to a downstream decision-maker instead of being classified directly.
• Condition fairness metrics (PPV, NPV, FPR, FNR) on non-deferred decisions to isolate fairness in active decisions.
• Use mass averaging and thresholding strategies to convert one accuracy profile (AP) into another while preserving fairness.
• Define conditional false positive and false negative rates by conditioning on non-deferral events to maintain consistency with standard definitions.
• Evaluate performance on the COMPAS dataset using thresholding with deferrals and AP transformation techniques.

Experimental results

Research questions

• RQ1 Can a calibrated soft classifier be post-processed using a single global threshold to equalize PPV or NPV across protected groups?
• RQ2 Under what conditions can group-aware thresholding equalize PPV or NPV without violating calibration?
• RQ3 Is it possible to simultaneously equalize PPV, NPV, FPR, and FNR across groups using post-processing with deferrals?
• RQ4 How do different deferral models affect the fairness and error rates of the resulting decision system?
• RQ5 What trade-offs exist between fairness, deferral rate, and accuracy in real-world applications like COMPAS?

Key findings

• There is no general way to post-process a calibrated classifier using a single global threshold to equalize PPV or NPV across protected groups.
• For certain well-behaved accuracy profiles (APs), group-aware thresholds can equalize either PPV or NPV, but not both simultaneously under global thresholding.
• With deferrals allowed, it is possible to equalize PPV, NPV, FPR, and FNR across groups for non-deferred decisions, effectively circumventing known impossibility results.
• Deferral-based post-processing enables full fairness equalization across multiple metrics, even when thresholding alone fails.
• Experiments on the COMPAS dataset show that deferral-based methods can achieve fairer outcomes with controlled deferral rates.
• The choice of deferral model significantly impacts fairness metrics, with conditional and unconditional error rate definitions yielding different interpretations of system performance.