[Paper Review] Impossibility results for fair representations

Tosca Lechner, Shai Ben-David+2 more|arXiv (Cornell University)|Jul 7, 2021

Ethics and Social Impacts of AI14 references5 citations

TL;DR

This paper proves fundamental impossibility results for fair data representations in machine learning: no representation can guarantee demographic parity or odds equality fairness across different tasks, especially when data distributions shift. Even with known marginal distributions, fairness and accurate classification cannot coexist for multiple tasks using a single representation, undermining the feasibility of transferable fairness solutions proposed in prior work.

ABSTRACT

With the growing awareness to fairness in machine learning and the realization of the central role that data representation has in data processing tasks, there is an obvious interest in notions of fair data representations. The goal of such representations is that a model trained on data under the representation (e.g., a classifier) will be guaranteed to respect some fairness constraints. Such representations are useful when they can be fixed for training models on various different tasks and also when they serve as data filtering between the raw data (known to the representation designer) and potentially malicious agents that use the data under the representation to learn predictive models and make decisions. A long list of recent research papers strive to provide tools for achieving these goals. However, we prove that this is basically a futile effort. Roughly stated, we prove that no representation can guarantee the fairness of classifiers for different tasks trained using it; even the basic goal of achieving label-independent Demographic Parity fairness fails once the marginal data distribution shifts. More refined notions of fairness, like Odds Equality, cannot be guaranteed by a representation that does not take into account the task specific labeling rule with respect to which such fairness will be evaluated (even if the marginal data distribution is known a priory). Furthermore, except for trivial cases, no representation can guarantee Odds Equality fairness for any two different tasks, while allowing accurate label predictions for both. While some of our conclusions are intuitive, we formulate (and prove) crisp statements of such impossibilities, often contrasting impressions conveyed by many recent works on fair representations.

Motivation & Objective

• To investigate whether generic data representations can guarantee fairness across multiple downstream classification tasks.
• To examine the feasibility of achieving demographic parity and odds equality fairness through fixed representations independent of task-specific data distributions.
• To resolve the discrepancy between claims of transferable fair representations in recent literature and the theoretical limitations of such approaches.
• To clarify the conceptual foundations of fairness in representation learning, particularly the role of data distribution and labeling rules.
• To demonstrate that fairness of individual features is ill-defined without considering the full feature set, challenging common assumptions in fairness literature.

Proposed method

• Formalizing fairness notions such as demographic parity (DP) and odds equality (EO) in terms of conditional independence constraints on group membership, predictions, and labels.
• Proving that no representation can ensure DP fairness across all classifiers when the marginal data distribution shifts between tasks.
• Demonstrating that for any two distinct tasks with the same marginal distribution but different labeling rules, no representation can simultaneously enable accurate classifiers and guarantee EO fairness.
• Introducing the concept of (ε,η)-fairness-enabling representations and showing that combining features can enable fairness even when individual features do not.
• Analyzing predictive rate parity (PRP) fairness and proving that adversarial fairness under PRP is only possible if success rates are equal across groups.
• Using counterexamples and formal proofs to show that fairness cannot be ensured without access to the data distribution or ground truth labels at evaluation time.

Experimental results

Research questions

• RQ1 Can a single data representation guarantee demographic parity fairness across all downstream classifiers, even when the marginal data distribution changes?
• RQ2 Is it possible to design a representation that ensures odds equality fairness for two different tasks with the same marginal data distribution but different labeling rules?
• RQ3 Can a representation guarantee fairness without access to the true labels or data distribution at test time?
• RQ4 What is the relationship between the fairness of individual features and the fairness of the full representation?
• RQ5 Under what conditions is adversarial fairness with respect to predictive rate parity achievable?

Key findings

• No data representation can guarantee demographic parity fairness for all classifiers when the marginal data distribution shifts between tasks, even if the distribution is known in advance.
• For any two different non-redundant tasks with the same marginal data distribution, no representation can simultaneously allow accurate label prediction and guarantee odds equality fairness for both tasks.
• Even with access to the true labels and marginal distribution, fairness cannot be universally guaranteed across tasks unless the representation is specifically tailored to each task.
• The fairness of a single feature is ill-defined; a feature may be fair in isolation but contribute to unfairness in a full representation due to interactions with other features.
• Adversarial fairness under predictive rate parity is only possible if the success rates of both groups are equal under the data distribution.
• The combination of features can enable fairness even when individual features or subsets are not fairness-enabling, highlighting the non-additive nature of fairness in representations.