[Paper Review] Fairness Without Demographics in Repeated Loss Minimization

Tatsunori Hashimoto, Megha Srivastava+2 more|arXiv (Cornell University)|Jun 20, 2018

Retirement, Disability, and EmploymentSocial Sciences108 citations

TL;DR

The paper shows that empirical risk minimization can amplify representation disparity over time when group identities are unknown, and introduces distributionally robust optimization to bound the worst-case risk across latent groups, improving minority outcomes in both simulated and real-world tasks.

ABSTRACT

Machine learning models (e.g., speech recognizers) are usually trained to minimize average loss, which results in representation disparity---minority groups (e.g., non-native speakers) contribute less to the training objective and thus tend to suffer higher loss. Worse, as model accuracy affects user retention, a minority group can shrink over time. In this paper, we first show that the status quo of empirical risk minimization (ERM) amplifies representation disparity over time, which can even make initially fair models unfair. To mitigate this, we develop an approach based on distributionally robust optimization (DRO), which minimizes the worst case risk over all distributions close to the empirical distribution. We prove that this approach controls the risk of the minority group at each time step, in the spirit of Rawlsian distributive justice, while remaining oblivious to the identity of the groups. We demonstrate that DRO prevents disparity amplification on examples where ERM fails, and show improvements in minority group user satisfaction in a real-world text autocomplete task.

Motivation & Objective

• Motivate representation disparity and its amplification in sequential learning with unknown group labels.
• Propose a DRO-based approach to bound the worst-case risk across latent groups and stabilize minority outcomes over time.
• Provide theoretical guarantees linking DRO to per-group risk control and retention dynamics.
• Demonstrate empirical benefits of DRO on simulated tasks and a real-world autocomplete experiment.

Proposed method

• Model data as coming from K latent groups with unknown identities and aim to minimize the worst-case group risk across time.
• Adopt a chi-square divergence ball around the empirical distribution and define the distributionally robust risk R_dro(θ; r) to upweight high-loss examples.
• Show that R_k(θ) ≤ R_dro(θ; r_k) for each group with r_k = (1/α_k − 1)^2, and use r_max = (1/α_min − 1)^2 to bound the overall worst-case risk.
• Provide a dual formulation where the DRO objective can be optimized via a surrogate F(θ; η) involving an η parameter and the (ℓ(θ; Z) − η)_+ term.
• Outline an optimization routine that alternates between θ updates (via SGD on the dual objective) and η selection (e.g., binary search), yielding a practical DRO solver.
• Discuss stability implications showing how DRO can yield a more robust, minority-favorable fixed point compared to ERM.

Experimental results

Research questions

• RQ1 Does ERM cause disparity amplification over time when demographic identities are unavailable?
• RQ2 Can DRO provide theoretical and empirical bounds on worst-case group risk and thereby stabilize minority outcomes over sequential updates?
• RQ3 Do simulated and real-world (e.g., autocomplete) tasks show DRO improving minority satisfaction and retention compared to ERM?
• RQ4 Under what conditions does ensuring a minimum minority proportion α_min lead to control of R_max over time when using DRO?

Key findings

• ERM can be unfair over time due to disparity amplification and unstable fixed points.
• DRO bounds the worst-case group risk and, with a lower bound on minority proportions, controls the worst-case risk across time.
• In simulations, DRO prevents minority accuracy collapse and maintains balanced performance across groups.
• In a Mechanical Turk autocomplete study, DRO improves minority satisfaction from 3.7 to 4.0 and minority retention from 0.7 to 0.85.
• DRO increases minority user counts through improved satisfaction and retention, illustrating practical benefits.