[Paper Review] Reranking individuals: The effect of fair classification within-groups

Sofie Goethals, Favier, Marco+1 more|arXiv (Cornell University)|Jan 24, 2024

Health Systems, Economic Evaluations, Quality of LifeEconomics, Econometrics and Finance3 citations

TL;DR

This paper challenges the conventional focus on between-group fairness metrics in bias mitigation, proposing a paradigm shift toward optimizing within-group ranking accuracy using model prediction scores before applying fairness constraints. It demonstrates that relying on post-hoc label adjustments leads to unrealistic positive rates and poor real-world applicability, advocating instead for selecting top-ranked individuals from each subgroup based on AUC-optimized rankings to achieve both fairness and practical feasibility.

ABSTRACT

Artificial Intelligence (AI) finds widespread application across various domains, but it sparks concerns about fairness in its deployment. The prevailing discourse in classification often emphasizes outcome-based metrics comparing sensitive subgroups without a nuanced consideration of the differential impacts within subgroups. Bias mitigation techniques not only affect the ranking of pairs of instances across sensitive groups, but often also significantly affect the ranking of instances within these groups. Such changes are hard to explain and raise concerns regarding the validity of the intervention. Unfortunately, these effects remain under the radar in the accuracy-fairness evaluation framework that is usually applied. Additionally, we illustrate the effect of several popular bias mitigation methods, and how their output often does not reflect real-world scenarios.

Motivation & Objective

• To address the limitation of existing bias mitigation evaluations that focus solely on between-group fairness metrics, often ignoring within-group ranking changes.
• To highlight that current benchmarking practices compare methods under unrealistic positive rate conditions, leading to misleading conclusions about performance.
• To argue that prediction scores—not just labels—should be the basis for evaluating and selecting individuals in fairness-aware machine learning.
• To propose a two-step framework: first, optimize subgroup ranking accuracy via AUC; second, select top-ranked individuals subject to fairness and capacity constraints.
• To demonstrate that post-hoc label adjustments often result in infeasible positive rates, undermining real-world usability.

Proposed method

• Propose a two-stage evaluation framework: first, assess bias mitigation methods based on their ability to produce the most accurate ranking within each subgroup using AUC as the metric.
• Use prediction scores (not final labels) to rank individuals within each sensitive group, ensuring subgroup-specific precision is prioritized.
• Apply fairness constraints (e.g., demographic parity) only after establishing the optimal within-group ranking, using the top-ranked individuals from each group.
• Introduce a constraint-aware selection process that respects real-world limits on positive rates (e.g., loan quotas, admission caps).
• Compare mitigation methods not by their final labels, but by the quality of their underlying score-based rankings, avoiding apples-to-oranges comparisons.
• Demonstrate that postprocessing methods preserve original rankings, while inprocessing and preprocessing methods often distort them, affecting within-group fairness.

Experimental results

Research questions

• RQ1 How do different bias mitigation methods affect the internal ranking of individuals within each sensitive group?
• RQ2 Why is relying solely on between-group fairness metrics insufficient for evaluating bias mitigation techniques in real-world applications?
• RQ3 To what extent do current benchmarking practices mislead due to comparing methods with vastly different positive rates?
• RQ4 Can a two-stage framework—first optimizing within-group ranking accuracy, then applying fairness constraints—lead to more realistic and fair outcomes?
• RQ5 How do prediction scores compare to final labels in guiding fair and practical decision-making in constrained environments?

Key findings

• The Adult Income dataset shows that bias mitigation methods can produce positive rates ranging from 0.5% to 39.3%, while the actual rate is 23.9%, highlighting the risk of comparing methods under incommensurate conditions.
• Postprocessing methods preserve the original ranking of instances within groups, whereas preprocessing and inprocessing methods often alter these rankings, potentially harming within-group fairness.
• The AUC metric effectively captures the quality of within-group ranking, making it a superior evaluation criterion than accuracy or fairness metrics applied to final labels.
• Relying on final prediction labels for comparison leads to misleading conclusions, as methods with the same accuracy may operate at vastly different positive rates, rendering them inapplicable in real-world settings.
• The proposed framework prevents underutilization of capacity and avoids unrealistically high or low positive rates, aligning better with practical constraints such as loan quotas or admission caps.
• Even when postprocessing methods are used, the fairness outcome depends on the quality of the underlying score-based ranking, which is often overlooked in standard evaluation.