[Paper Review] Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations

Alex Beutel, Jilin Chen+2 more|arXiv (Cornell University)|Jul 1, 2017

Adversarial Robustness in Machine LearningComputer Science11 references295 citations

TL;DR

The paper uses adversarial training to learn fair latent representations without access to sensitive attributes at inference, showing how adversarial data distribution shapes fairness definitions and that small, balanced adversarial datasets can meaningfully improve fairness with trade-offs in accuracy.

ABSTRACT

How can we learn a classifier that is "fair" for a protected or sensitive group, when we do not know if the input to the classifier belongs to the protected group? How can we train such a classifier when data on the protected group is difficult to attain? In many settings, finding out the sensitive input attribute can be prohibitively expensive even during model training, and sometimes impossible during model serving. For example, in recommender systems, if we want to predict if a user will click on a given recommendation, we often do not know many attributes of the user, e.g., race or age, and many attributes of the content are hard to determine, e.g., the language or topic. Thus, it is not feasible to use a different classifier calibrated based on knowledge of the sensitive attribute. Here, we use an adversarial training procedure to remove information about the sensitive attribute from the latent representation learned by a neural network. In particular, we study how the choice of data for the adversarial training effects the resulting fairness properties. We find two interesting results: a small amount of data is needed to train these adversarial models, and the data distribution empirically drives the adversary's notion of fairness.

Motivation & Objective

• Motivate learning fair predictions when sensitive attributes are unavailable at inference or hard to label during training.
• Link fairness definitions to the data distribution used for the adversarial objective.
• Empirically assess how much adversarial data is needed and how its distribution impacts fairness outcomes.
• Demonstrate the trade-offs between model accuracy and fairness under different adversarial data regimes.

Proposed method

• Propose a multi-head neural network where one head predicts Y and an adversary tries to predict a sensitive attribute Z from the latent g(X).
• Introduce a negative-gradient trick via J_lambda to ensure g(X) hides information about Z while still allowing Y to be predicted well.
• Analyze how the dataset S (used by the adversary) and its distribution over Y and Z affect the resulting fairness properties.
• Experiment with varying sizes and distributions of S to study effects on fairness metrics and accuracy.

Experimental results

Research questions

• RQ1 How does the choice and distribution of adversarial training data S influence the resulting fairness of the model?
• RQ2 What is the impact of using balanced versus unbalanced distributions of the sensitive attribute Z in adversarial training on fairness and accuracy?
• RQ3 How much adversarial data is needed to achieve meaningful improvements in fairness without sacrificing too much accuracy?
• RQ4 How do different distributions over the primary label Y (income) affect equality of opportunity and demographic parity in the learned representations?

Key findings

• Balanced adversarial data substantially improves fairness metrics and stabilizes training, though at some cost to accuracy.
• Adversarial data distributions aligned with high-income or low-income groups yield improvements specific to equality of opportunity for the corresponding group, while mixing groups improves fairness across metrics.
• Even very small adversarial datasets (as few as 500 examples) can produce meaningful fairness enhancements.
• The distribution over the primary label Y in the adversarial data drives different fairness outcomes, consistent with theoretical expectations linking data choice to fairness definitions.
• A balanced Z distribution in adversarial training generally yields stronger fairness effects than random sampling from the natural distribution.
• There is a trade-off: stronger fairness (lower disparity) often accompanies reduced predictive accuracy, with lambda tuning mediating this balance.