[Paper Review] Is There a Trade-Off Between Fairness and Accuracy? A Perspective Using Mismatched Hypothesis Testing

Sanghamitra Dutta, Dennis Wei+4 more|arXiv (Cornell University)|Oct 17, 2019

Ethics and Social Impacts of AI48 citations

TL;DR

The paper reframes fairness-accuracy trade-offs via mismatched hypothesis testing and Chernoff information, showing no inherent trade-off under ideal distributions and providing criteria to alleviate trade-offs in practice.

ABSTRACT

A trade-off between accuracy and fairness is almost taken as a given in the existing literature on fairness in machine learning. Yet, it is not preordained that accuracy should decrease with increased fairness. Novel to this work, we examine fair classification through the lens of mismatched hypothesis testing: trying to find a classifier that distinguishes between two ideal distributions when given two mismatched distributions that are biased. Using Chernoff information, a tool in information theory, we theoretically demonstrate that, contrary to popular belief, there always exist ideal distributions such that optimal fairness and accuracy (with respect to the ideal distributions) are achieved simultaneously: there is no trade-off. Moreover, the same classifier yields the lack of a trade-off with respect to ideal distributions while yielding a trade-off when accuracy is measured with respect to the given (possibly biased) dataset. To complement our main result, we formulate an optimization to find ideal distributions and derive fundamental limits to explain why a trade-off exists on the given biased dataset. We also derive conditions under which active data collection can alleviate the fairness-accuracy trade-off in the real world. Our results lead us to contend that it is problematic to measure accuracy with respect to data that reflects bias, and instead, we should be considering accuracy with respect to ideal, unbiased data.

Motivation & Objective

• Motivate the fairness-accuracy question and challenge the assumed trade-off in real data.
• Introduce separability via Chernoff information to quantify accuracy and fairness for groups.
• Show that biased mappings to observed data can create apparent trade-offs.
• Propose ideal distributions under which fairness and accuracy align and provide construction methods.
• Derive conditions under which active data collection reduces or eliminates the trade-off.

Proposed method

• Model binary classification under a construct space and a biased observed space with protected attribute Z.
• Use likelihood ratio detectors and Chernoff exponents to quantify error probabilities for each group.
• Define separability as Chernoff information between P0/P1 and Q0/Q1 for unprivileged vs privileged groups.

Experimental results

Research questions

• RQ1 Does a real-world accuracy-fairness trade-off arise from biased mappings between construct and observed spaces?
• RQ2 Can there exist ideal distributions where fairness and accuracy are simultaneously maximized?
• RQ3 Under what data-collection conditions does increasing features improve separability and reduce the trade-off?
• RQ4 How can one construct ideal distributions that preserve fairness while improving accuracy on the ideal data?

Key findings

• Chernoff information is used as a separability measure to quantify the accuracy-fairness trade-off for each group.
• If C(P0,P1) < C(Q0,Q1), the Bayes optimal detectors are unfair on the observed data, and any fairness adjustment lowers accuracy for at least one group (Theorem 1).
• There exist ideal distributions for the unprivileged group such that the Bayes optimal detector is fair on the given data and optimal on the ideal data (Theorem 2).
• An optimization framework can yield ideal distributions that minimize divergence from the observed data while achieving fairness and matching the privileged group’s separability on the ideal data (Theorem 2; optimization (4)).
• Active data collection can alleviate the trade-off by increasing separability (Theorem 3).
• The work argues accuracy should be evaluated with respect to ideal, unbiased data rather than biased observed data.