[Paper Review] A Reductions Approach to Fair Classification

Alekh Agarwal, Alina Beygelzimer+3 more|arXiv (Cornell University)|Mar 6, 2018

Ethics and Social Impacts of AISocial Sciences24 references111 citations

TL;DR

The paper presents a general method to enforce fairness in binary classification by reducing fair classification to a sequence of cost-sensitive classification problems, enabling a randomized classifier with minimum empirical error under fairness constraints. It treats the classifier as a black box and provides finite-sample guarantees across various fairness definitions.

ABSTRACT

We present a systematic approach for achieving fairness in a binary classification setting. While we focus on two well-known quantitative definitions of fairness, our approach encompasses many other previously studied definitions as special cases. The key idea is to reduce fair classification to a sequence of cost-sensitive classification problems, whose solutions yield a randomized classifier with the lowest (empirical) error subject to the desired constraints. We introduce two reductions that work for any representation of the cost-sensitive classifier and compare favorably to prior baselines on a variety of data sets, while overcoming several of their disadvantages.

Motivation & Objective

• Motivate and formalize fairness in binary classification with respect to a protected attribute.
• Provide a reduction from fair classification to cost-sensitive learning that works with any classifier family.
• Allow randomized classifiers to achieve the best accuracy under fairness constraints.
• Offer finite-sample guarantees and practical algorithms for achieving fairness without requiring test-time protected attributes.

Proposed method

• Formulate fairness as linear inequalities on conditional moments: M mu(h) <= c, capturing DP and EO as special cases.
• Reduce the constrained empirical problem to a saddle-point formulation using Lagrange multipliers and a no-regret optimization framework.
• Use an exponential gradient (Exponential Weights) algorithm for the lambda-player and best-response cost-sensitive classifiers for the h-player.
• Express the best-response for the h-player as a cost-sensitive classification problem with constructed costs that incorporate the current multipliers.
• Provide finite-sample guarantees linking suboptimality, statistical error via Rademacher complexity, and fairness constraint violation.
• Offer guidance for grid-search over multipliers for obtaining deterministic classifiers when desired.

Experimental results

Research questions

• RQ1 How can binary classification be performed under formal fairness constraints such as DP and EO without restricting the classifier family?
• RQ2 Can a black-box classifier be reused within a reduction framework to achieve the lowest empirical error under fairness constraints?
• RQ3 What are the finite-sample guarantees for accuracy and fairness when solving the reduced cost-sensitive problems?
• RQ4 How can the fairness constraints be effectively incorporated into cost-sensitive learning via Lagrange multipliers?
• RQ5 What is the impact of using randomized versus deterministic classifiers in achieving the accuracy–fairness tradeoff?

Key findings

• A sequence of cost-sensitive classification problems can realize the lowest empirical error subject to fairness constraints.
• A saddle-point algorithm with Exponential Gradient for the lambda-player achieves a nu-approximate saddle point with sublinear suboptimality.
• The approach covers many definitions of fairness that can be expressed as linear inequalities on conditional moments, including DP and EO.
• Theoretical guarantees link statistical error (via Rademacher complexity) and optimization error to the final classifier performance.
• Examples show how DP and EO lead to specific cost constructions and iteration bounds under the framework.