[Paper Review] On Adversarial Bias and the Robustness of Fair Machine Learning

Hongyan Chang, Ta Duy Nguyen+3 more|arXiv (Cornell University)|Jun 15, 2020

Adversarial Robustness in Machine LearningComputer Science37 references37 citations

TL;DR

This paper analyzes how data poisoning attacks targeting fairness constraints (specifically equalized odds) can degrade both accuracy and fairness, and presents attack algorithms that exploit the robustness-fairness trade-off.

ABSTRACT

Optimizing prediction accuracy can come at the expense of fairness. Towards minimizing discrimination against a group, fair machine learning algorithms strive to equalize the behavior of a model across different groups, by imposing a fairness constraint on models. However, we show that giving the same importance to groups of different sizes and distributions, to counteract the effect of bias in training data, can be in conflict with robustness. We analyze data poisoning attacks against group-based fair machine learning, with the focus on equalized odds. An adversary who can control sampling or labeling for a fraction of training data, can reduce the test accuracy significantly beyond what he can achieve on unconstrained models. Adversarial sampling and adversarial labeling attacks can also worsen the model's fairness gap on test data, even though the model satisfies the fairness constraint on training data. We analyze the robustness of fair machine learning through an empirical evaluation of attacks on multiple algorithms and benchmark datasets.

Motivation & Objective

• Motivate and quantify the tension between fairness and robustness in ML under data poisoning.
• Analyze how attackers can degrade both accuracy and fairness by manipulating training data.
• Develop attack strategies against fair models and empirically evaluate their impact on benchmark datasets.

Proposed method

• Formulates a bi-level optimization to maximize training loss while satisfying a fairness constraint under equalized odds.
• Introduce tractable approximations: replace the inner optimization with a loss surrogate and use a Lagrangian penalty for the fairness constraint.
• Propose online gradient-based poisoning algorithms (Algorithms 1 and 2) that select poisoning points to maximize a weighted sum of loss and fairness gap.
• Approximate the fairness gap with an additive surrogate to enable data-point–level poisoning decisions.
• Evaluate attacks on fair logistic regression models using post-processing and reductions approaches on COMPAS and Adult datasets.
• Provide theoretical guarantees linking the attack performance to the optimal poisoning solution under the approximations.

Experimental results

Research questions

• RQ1 Does enforcing equalized odds fairness constrain robustness to data poisoning compared to unconstrained models?
• RQ2 How does adversarial sampling and labeling of training data affect test accuracy and fairness generalization?
• RQ3 Can attacker strategies exploit the fairness constraint to disproportionately harm minority/underrepresented groups?
• RQ4 What is the impact of fairness level (δ) on robustness under adversarial bias?

Key findings

• Fairness constraints can significantly reduce robustness to data poisoning compared to unconstrained models.
• Under attack, fair models can suffer large test accuracy losses, sometimes approaching the performance of constant predictors.
• Stronger fairness (smaller δ) increases vulnerability to poisoning, widening the accuracy gap under attack.
• Attacks tend to place poisoned data in the smallest subgroup with the least frequent label, exploiting the fairness weighting.
• Adversarial bias can worsen the fairness gap on test data, even when training data fairness is satisfied.
• Poisoning that targets sampling (and labeling) can cause fair models to generalize worse in fairness than unconstrained models.