[Paper Review] There is no trade-off: enforcing fairness can improve accuracy

Subha Maity, Debarghya Mukherjee+2 more|arXiv (Cornell University)|May 4, 2021

Explainable Artificial Intelligence (XAI)7 references4 citations

TL;DR

This paper challenges the widely held belief that fairness and model performance are in conflict, demonstrating that enforcing fairness can improve accuracy when biases in training data stem from sampling imbalances. By aligning model predictions with the Bayes decision rule on unbiased test data, fairness constraints can correct for data bias and enhance generalization.

ABSTRACT

One of the main barriers to the broader adoption of algorithmic fairness in machine learning is the trade-off between fairness and performance of ML models: many practitioners are unwilling to sacrifice the performance of their ML model for fairness. In this paper, we show that this trade-off may not be necessary. If the algorithmic biases in an ML model are due to sampling biases in the training data, then enforcing algorithmic fairness may improve the performance of the ML model on unbiased test data. We study conditions under which enforcing algorithmic fairness helps practitioners learn the Bayes decision rule for (unbiased) test data from biased training data. We also demonstrate the practical implications of our theoretical results in real-world ML tasks.

Motivation & Objective

• To investigate whether enforcing algorithmic fairness can improve model performance on unbiased test data.
• To analyze conditions under which fairness constraints help recover the Bayes decision rule from biased training data.
• To demonstrate the practical benefits of fairness enforcement in real-world machine learning tasks.
• To challenge the common assumption that fairness comes at the cost of performance.

Proposed method

• The authors analyze the impact of fairness constraints on model learning when training data exhibits sampling biases.
• They derive conditions under which enforcing fairness leads to better alignment with the Bayes optimal decision rule on unbiased test data.
• The method involves theoretical analysis of how fairness constraints correct for data distribution shifts caused by biased sampling.
• The approach uses statistical learning theory to show that fairness enforcement can reduce generalization error on unbiased data.
• The framework is validated through empirical evaluation on real-world machine learning tasks.
• The core idea is that fairness constraints act as regularization that corrects for data bias, improving generalization.

Experimental results

Research questions

• RQ1 Under what conditions does enforcing fairness improve model accuracy on unbiased test data?
• RQ2 Can fairness constraints help recover the Bayes decision rule when training data is biased due to sampling?
• RQ3 Does enforcing fairness reduce generalization error in the presence of data sampling bias?
• RQ4 How does fairness enforcement compare to standard training in terms of performance on unbiased data?
• RQ5 What are the practical implications of using fairness constraints to correct for data bias in real-world ML applications?

Key findings

• Enforcing fairness can improve model accuracy when training data contains sampling biases, contradicting the assumption of a fairness-performance trade-off.
• Fairness constraints help the model converge to the Bayes decision rule on unbiased test data by correcting for data distribution shifts.
• The improvement in accuracy arises because fairness acts as a form of regularization that mitigates the effects of biased sampling.
• Empirical results in real-world tasks confirm that fairness enforcement enhances performance on unbiased test sets.
• The theoretical conditions under which fairness improves accuracy are derived and shown to be practically relevant in real-world scenarios.
• The study demonstrates that fairness is not a cost but a corrective mechanism that enhances model generalization when data is biased.