[Paper Review] Fairness-Aware PAC Learning from Corrupted Data

Nikola Konstantinov, Christoph H. Lampert|arXiv (Cornell University)|Feb 11, 2021

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper investigates the fundamental limits of fairness-aware PAC learning under adversarial data corruption, showing that even with infinite data, no learning algorithm can guarantee fairness when an adversary corrupts a fraction of the training data. It proves that excess bias in fairness metrics grows inversely with the frequency of underrepresented protected groups, and establishes order-optimal guarantees for two natural learning algorithms under the malicious adversary model, demonstrating tightness of the theoretical bounds up to constant factors.

ABSTRACT

Addressing fairness concerns about machine learning models is a crucial step towards their long-term adoption in real-world automated systems. While many approaches have been developed for training fair models from data, little is known about the robustness of these methods to data corruption. In this work we consider fairness-aware learning under worst-case data manipulations. We show that an adversary can in some situations force any learner to return an overly biased classifier, regardless of the sample size and with or without degrading accuracy, and that the strength of the excess bias increases for learning problems with underrepresented protected groups in the data. We also prove that our hardness results are tight up to constant factors. To this end, we study two natural learning algorithms that optimize for both accuracy and fairness and show that these algorithms enjoy guarantees that are order-optimal in terms of the corruption ratio and the protected groups frequencies in the large data limit.

Motivation & Objective

• To investigate the robustness of fairness-aware machine learning under worst-case data corruption.
• To analyze whether fairness guarantees can be preserved when an adversary manipulates training data adversarially.
• To determine the fundamental limits of fairness-aware learning in the presence of corrupted data, especially for underrepresented protected groups.
• To establish tight theoretical bounds on excess bias and corruption resilience in fairness learning.
• To evaluate the performance of natural fairness-optimized learning algorithms under adversarial corruption, proving order-optimality in the large-sample limit.

Proposed method

• Models data corruption via a malicious adversary who replaces a fraction of training points with arbitrary data, with full knowledge of the learning algorithm and data distribution.
• Focuses on two standard fairness constraints: demographic parity and equal opportunity in binary classification.
• Uses PAC learning theory to derive generalization bounds under adversarial corruption, analyzing both accuracy and fairness simultaneously.
• Applies concentration inequalities and union bounds to derive high-probability bounds on empirical risk and fairness deviation under corruption.
• Proves that hardness results are tight up to constant factors by constructing matching upper bounds for two natural learning algorithms.
• Analyzes the impact of protected group frequency on irreducible fairness bias, showing inverse proportionality to the rarer group's frequency.

Experimental results

Research questions

• RQ1 Can fairness-aware learning algorithms guarantee fairness under arbitrary, adversarial data corruption?
• RQ2 What is the fundamental limit of fairness in learning when the training data is corrupted by a malicious adversary?
• RQ3 How does the frequency of underrepresented protected groups affect the irreducible bias in fairness metrics under corruption?
• RQ4 Are there learning algorithms that achieve order-optimal performance in terms of corruption ratio and group frequency in the large-data limit?
• RQ5 Can the excess bias introduced by data corruption be bounded, and are these bounds tight up to constant factors?

Key findings

• No fairness-aware learning algorithm can guarantee convergence to the accuracy-fairness Pareto front under adversarial data corruption, even with infinite data.
• The irreducible excess bias in fairness metrics is inversely proportional to the frequency of the rarer protected group, making underrepresented groups especially vulnerable.
• An adversary can force any learner to produce a highly biased classifier that is still accurate, potentially evading detection in privacy-preserving systems.
• The hardness results are tight up to constant factors, as demonstrated by matching upper bounds for two natural learning algorithms.
• Both algorithms achieve order-optimal performance in the large-data limit, with guarantees that scale optimally with respect to corruption ratio and protected group frequencies.
• The theoretical bounds show that fairness robustness degrades significantly when protected group frequencies are low, even under optimal learning strategies.