[Paper Review] Equalized odds postprocessing under imperfect group information

Pranjal Awasthi, Matthäus Kleindeßner+1 more|arXiv (Cornell University)|Jun 7, 2019

Ethics and Social Impacts of AI38 references17 citations

TL;DR

This paper studies the robustness of equalized odds postprocessing under imperfect protected attribute information, showing that fairness improvements are preserved when the protected attribute is perturbed—provided the perturbation satisfies certain natural conditions. It formally proves that under these conditions, the resulting classifier is fairer than the original and maintains low error, even with moderate perturbations.

ABSTRACT

Most approaches aiming to ensure a model's fairness with respect to a protected attribute (such as gender or race) assume to know the true value of the attribute for every data point. In this paper, we ask to what extent fairness interventions can be effective even when only imperfect information about the protected attribute is available. In particular, we study the prominent equalized odds postprocessing method of Hardt et al. (2016) under a perturbation of the attribute. We identify conditions on the perturbation that guarantee that the bias of a classifier is reduced even by running equalized odds with the perturbed attribute. We also study the error of the resulting classifier. We empirically observe that under our identified conditions most often the error does not suffer from a perturbation of the protected attribute. For a special case, we formally prove this observation to be true.

Motivation & Objective

• To investigate whether fairness interventions like equalized odds postprocessing remain effective when the protected attribute is not perfectly known.
• To identify conditions on attribute perturbations that still guarantee improved fairness over a baseline classifier.
• To analyze the trade-off between fairness and accuracy when using perturbed protected attributes in postprocessing.
• To formally establish that error remains comparable to the true-attribute case under mild perturbations, particularly in balanced settings.

Proposed method

• Analyzes the equalized odds postprocessing method of Hardt et al. (2016) under a model where the protected attribute is perturbed via independent flipping with probabilities γ₀ and γ₁.
• Derives conditions on the perturbation (γ₀, γ₁) under which the fairness bias of the resulting classifier is strictly reduced compared to the original classifier.
• Introduces a formal fairness bias measure and uses it to compare the fairness of classifiers trained with perturbed vs. true attributes.
• Empirically evaluates performance on real-world datasets (COMPAS, Adult) and compares with baseline methods, including DRO and standard ERM.
• Applies the method in a sequential classification setting to assess long-term fairness and accuracy under repeated training.
• Proves a formal result in the balanced case (uniform joint distribution over Y and A) showing that error is not significantly increased by perturbation.

Experimental results

Research questions

• RQ1 Under what conditions on the perturbation of the protected attribute does equalized odds postprocessing still reduce fairness bias compared to the original classifier?
• RQ2 How does the error of the classifier trained with perturbed attributes compare to the error of the classifier trained with the true attribute?
• RQ3 Is the fairness improvement from equalized odds postprocessing robust to moderate levels of attribute noise or misclassification?
• RQ4 Can equalized odds postprocessing serve as a reliable fairness method in settings where the protected attribute is only approximately known?
• RQ5 How does the performance of equalized odds with perturbed attributes compare to distributionally robust optimization (DRO) in sequential learning settings?

Key findings

• For γ < 0.5 in a balanced setting (uniform P(Y=y, A=a) = 1/4), the fairness bias of the postprocessed classifier is strictly smaller than that of the original classifier.
• Empirical results show that under the identified conditions, the error of the perturbed-attribute classifier is typically not larger than that of the true-attribute classifier, with differences being negligible for moderate perturbations.
• In the balanced case, the paper formally proves that the error of the classifier trained with perturbed attributes is not significantly higher than that of the classifier trained with the true attribute.
• The equalized odds postprocessing method maintains fairness improvements even when the protected attribute is flipped with high probability (e.g., γ₀ = γ₁ = 0.45), outperforming standard ERM in preventing disparity amplification.
• The method remains effective even when the perturbation is asymmetric (e.g., γ₀ = 0.1, γ₁ = 0.8), suggesting robustness to class-specific noise.
• The identified conditions on perturbation are necessary for guaranteeing fairness improvement, meaning no such guarantee holds if these conditions are violated.