[Paper Review] Removing biased data to improve fairness and accuracy

Sahil Verma, Michael D. Ernst+1 more|arXiv (Cornell University)|Feb 5, 2021

Ethics and Social Impacts of AI92 references10 citations

TL;DR

This paper proposes a black-box pre-processing method that improves fairness and accuracy in machine learning by identifying and removing the most biased training data points. Using influence functions to pinpoint data points responsible for discriminatory predictions, the approach reduces individual discrimination to near zero while boosting model accuracy across six real-world datasets.

ABSTRACT

Machine learning systems are often trained using data collected from historical decisions. If past decisions were biased, then automated systems that learn from historical data will also be biased. We propose a black-box approach to identify and remove biased training data. Machine learning models trained on such debiased data (a subset of the original training data) have low individual discrimination, often 0%. These models also have greater accuracy and lower statistical disparity than models trained on the full historical data. We evaluated our methodology in experiments using 6 real-world datasets. Our approach outperformed seven previous approaches in terms of individual discrimination and accuracy.

Motivation & Objective

• Address the pervasive issue of bias in machine learning models trained on historical data that reflect past discriminatory decisions.
• Reduce individual discrimination in model predictions without compromising accuracy, especially in high-stakes applications like hiring, lending, and criminal justice.
• Develop a pre-processing technique that is applicable to black-box models and does not require access to model architecture or gradients.
• Improve fairness metrics—individual discrimination and statistical disparity—while simultaneously increasing test accuracy, contrary to the common trade-off in fairness methods.

Proposed method

• Generate discriminatory pairs: identify similar individuals (differing only in sensitive attributes) who receive dissimilar predictions from the model.
• Apply influence functions to measure the contribution of each training data point to the dissimilar predictions made for discriminatory pairs.
• Rank training data points by their influence on discriminatory predictions, prioritizing those most responsible for unfair outcomes.
• Remove the top-k most influential data points (those most responsible for discrimination) to create a debiased training dataset.
• Retrain the model on the cleaned dataset to produce a fairer and more accurate model.
• Use a black-box approach that only requires access to the model’s train and predict functions, enabling application to proprietary models.

Experimental results

Research questions

• RQ1 Can identifying and removing the most influential training data points responsible for discriminatory predictions improve fairness without sacrificing accuracy?
• RQ2 How does the proposed method compare to existing fairness techniques in terms of individual discrimination, statistical disparity, and test accuracy?
• RQ3 Is it possible to achieve near-zero individual discrimination while simultaneously improving model accuracy through data debiasing?
• RQ4 Can influence functions effectively identify biased training instances even in black-box models without architectural access?
• RQ5 Does removing influential biased data points lead to better fairness-accuracy trade-offs than existing pre-processing, in-processing, or post-processing methods?

Key findings

• The proposed method reduced individual discrimination to 0% in most experiments, significantly outperforming seven prior approaches.
• The model trained on the debiased dataset achieved higher test accuracy than the baseline model trained on the full historical dataset in all eight experiments.
• The method improved both individual discrimination and statistical disparity metrics simultaneously, unlike most prior approaches that trade off fairness for accuracy.
• The technique is effective across diverse real-world datasets, including those related to credit, hiring, and criminal justice.
• The approach is the only one among eight tested that consistently improved all three metrics—individual discrimination, statistical disparity, and accuracy—regardless of the optimization goal.
• Influence functions successfully identified training data points most responsible for unfair predictions, enabling targeted removal without model retraining from scratch.