[Paper Review] Fairness without Demographics through Adversarially Reweighted Learning

Preethi Lahoti, Alex Beutel+6 more|arXiv (Cornell University)|Jun 23, 2020

Ethics and Social Impacts of AI52 references46 citations

TL;DR

ARL trains models to improve max-min fairness across unobserved protected groups by adversarially reweighting training examples, achieving better worst-case AUC than DRO while maintaining or improving overall AUC.

ABSTRACT

Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns. However, in practice factors like privacy and regulation often preclude the collection of protected features, or their use for training or inference, severely limiting the applicability of traditional fairness research. Therefore we ask: How can we train an ML model to improve fairness when we do not even know the protected group memberships? In this work we address this problem by proposing Adversarially Reweighted Learning (ARL). In particular, we hypothesize that non-protected features and task labels are valuable for identifying fairness issues, and can be used to co-train an adversarial reweighting approach for improving fairness. Our results show that {ARL} improves Rawlsian Max-Min fairness, with notable AUC improvements for worst-case protected groups in multiple datasets, outperforming state-of-the-art alternatives.

Motivation & Objective

• Motivate fairness in ML when protected attributes are unavailable due to privacy or regulation.
• Formalize Rawlsian Max-Min fairness for unknown protected groups.
• Introduce ARL to upweight computationally-identifiable error regions to improve worst-off group performance.
• Show empirical and analytical insights into ARL’s robustness and behavior across datasets.

Proposed method

• Formulate a minimax objective that maximizes the minimum group utility across unobserved protected groups.
• Introduce an adversarial reweighting mechanism where a predictor f_phi(X,Y) identifies high-loss regions and produces weights lambda_phi to weight losses.
• Constrain and normalize adversarial weights to ensure stability and participation of all examples.
• Train a learner h_theta and an adversary f_phi alternatively in a two-player game (minimax optimization).
• Use a linear adversary or simple networks for small datasets; provide open-source TensorFlow implementation.

Experimental results

Research questions

• RQ1 Can fairness be improved when protected group memberships are unknown at training and inference time?
• RQ2 Does adversarially reweighting based on computationally-identifiable error regions yield superior worst-case (AUC_min) performance compared with existing methods like DRO?
• RQ3 What is ARL’s impact on overall utility (AUC_avg) and robustness to representation and label bias?
• RQ4 How do learned example weights correlate with actual misclassifications and base-rate shifts?

Key findings

• ARL improves worst-case protected-group AUC across Adult, LSAC, and COMPAS datasets, outperforming DRO and baseline on AUC_min and often on AUC_minority.
• ARL yields up to 6.5 percentage points improvement over baseline in AUC for Adult, and improves AUC_avg and AUC_macro on several datasets.
• ARL achieves better or comparable overall AUC (AUC_avg) than baselines and DRO, indicating a favorable utility-fairness trade-off.
• In experiments, ARL’s weights focus on misclassified or high-loss regions and adapt to base-rate changes, demonstrating meaningful, interpretable weighting patterns.
• ARL remains robust to representation bias and certain label-noise scenarios, but can degrade with heavy label noise, similar to other fairness approaches.