[Paper Review] Identifying and Correcting Label Bias in Machine Learning

Heinrich Jiang, Ofir Nachum|arXiv (Cornell University)|Jan 15, 2019

Ethics and Social Impacts of AI41 references116 citations

TL;DR

The paper formulates a model of how biased labels arise and proposes a re-weighting scheme that, without altering labels, yields unbiased classifiers across various fairness notions. It provides theoretical guarantees and empirical validation on standard fairness datasets.

ABSTRACT

Datasets often contain biases which unfairly disadvantage certain groups, and classifiers trained on such datasets can inherit these biases. In this paper, we provide a mathematical formulation of how this bias can arise. We do so by assuming the existence of underlying, unknown, and unbiased labels which are overwritten by an agent who intends to provide accurate labels but may have biases against certain groups. Despite the fact that we only observe the biased labels, we are able to show that the bias may nevertheless be corrected by re-weighting the data points without changing the labels. We show, with theoretical guarantees, that training on the re-weighted dataset corresponds to training on the unobserved but unbiased labels, thus leading to an unbiased machine learning classifier. Our procedure is fast and robust and can be used with virtually any learning algorithm. We evaluate on a number of standard machine learning fairness datasets and a variety of fairness notions, finding that our method outperforms standard approaches in achieving fair classification.

Motivation & Objective

• Motivate and formalize how label bias arises from biased labeling processes while aiming for unbiased ground-truth outcomes.
• Propose a data re-weighting technique to recover unbiased labels without modifying observed labels or features.
• Provide theoretical guarantees showing equivalence between re-weighted biased-label training and unbiased-label training.
• Develop an algorithm to estimate bias coefficients and apply them to training with common classifiers.
• Demonstrate effectiveness across multiple fairness notions and benchmark datasets.

Proposed method

• Assume an unknown unbiased ground-truth label function and a biased observed label function related through a constrained optimization with KL-divergence.
• Derive a closed-form relation: y_bias is proportional to y_true times exp(-sum_k lambda_k c_k(x,y)).
• Show that y_true is proportional to y_bias times exp(+sum_k lambda_k c_k(x,y)).
• Propose a weighting w(x,y) = exp(sum_k lambda_k c_k(x,y)) normalized by sum_y w(x,y) to re-weight training examples.
• Prove that training with weights on biased labels is equivalent to training on true labels under a distribution tilde P.
• Provide an algorithm (Algorithm 1) to learn coefficients lambda_k and train with re-weighted loss to satisfy fairness constraints.

Experimental results

Research questions

• RQ1 How can label bias be mathematically modeled when true labels are unknown but fairness constraints are desired?
• RQ2 Can a re-weighting of biased-labeled data recover training dynamics as if optimizing on unbiased labels?
• RQ3 How can bias coefficients be learned and updated to satisfy demographic parity, disparate impact, equal opportunity, and equalized odds?
• RQ4 What are the theoretical guarantees (rates of consistency) for learning with re-weighted biased data?
• RQ5 How does the proposed method perform across standard fairness datasets and notions compared to post-processing and Lagrangian approaches?

Key findings

• A closed-form expression links observed biased labels to the underlying unbiased labels via exponential weights tied to fairness constraints.
• The re-weighting scheme yields an equivalent training objective to learning with unbiased labels under a modified feature distribution, under mild conditions.
• An iterative Algorithm 1 can learn the bias coefficients and sample weights to achieve fairness notions such as demographic parity, disparate impact, and equal opportunity.
• Empirical evaluation on standard fairness datasets shows the proposed re-weighting method outperforming baseline approaches in fairness violations across several notions.
• Theoretical results establish finite-sample rates of consistency for the weighted estimator, with manifold-aware rates improving dimensionality dependence.