[Paper Review] Empirical Risk Minimization under Fairness Constraints

Michele Donini, Luca Oneto+3 more|arXiv (Cornell University)|Feb 23, 2018

Ethics and Social Impacts of AISocial Sciences42 references88 citations

TL;DR

This paper introduces Fair Empirical Risk Minimization (FERM), a framework to train classifiers under a fairness constraint that equalizes conditional risk across sensitive groups, with theoretical consistency guarantees and practical kernel/linear implementations.

ABSTRACT

We address the problem of algorithmic fairness: ensuring that sensitive variables do not unfairly influence the outcome of a classifier. We present an approach based on empirical risk minimization, which incorporates a fairness constraint into the learning problem. It encourages the conditional risk of the learned classifier to be approximately constant with respect to the sensitive variable. We derive both risk and fairness bounds that support the statistical consistency of our approach. We specify our approach to kernel methods and observe that the fairness requirement implies an orthogonality constraint which can be easily added to these methods. We further observe that for linear models the constraint translates into a simple data preprocessing step. Experiments indicate that the method is empirically effective and performs favorably against state-of-the-art approaches.

Motivation & Objective

• Motivate and formalize a fairness constraint that equalizes conditional risk across sensitive groups for positive labels.
• Develop a Fair Empirical Risk Minimization (FERM) framework that integrates the constraint into ERM.
• Provide theoretical risk and fairness consistency bounds for FERM.
• Show how fairness constraints translate to kernel methods (orthogonality constraint) and to a linear data preprocessing step.
• Demonstrate empirical effectiveness against state-of-the-art fairness methods across multiple datasets.

Proposed method

• Define an ε-fairness constraint based on the difference of positive-class risks across groups and embed it into ERM, yielding Fair ERM (FERM).
• Derive risk and fairness bounds to establish statistical consistency of FERM and discuss surrogate convex relaxations for practical optimization.
• Specialize to kernel methods: express the constraint as an inner-product bound in RKHS, leading to a constrained Tikhonov-regularized objective solvable via standard SVM-like solvers.
• Show that for linear models the ε-fair constraint reduces to a data preprocessing step that enforces fairness by removing discriminatory directions.
• Provide a theoretical link between convex surrogate FERM problems and the original non-convex fairness goal, including a condition (Δ̂) that bounds the gap to EO fairness.
• Include a practical validation procedure and a novel hyperparameter validation approach to balance accuracy and DEO (demographic/equal-opportunity measure).

Experimental results

Research questions

• RQ1 Can we formulate a fairness constraint that captures Equal Opportunity-like fairness as a bound on the conditional risk of the positive class across groups?
• RQ2 Does embedding a fairness constraint into ERM yield a statistically consistent estimator under standard learnability assumptions?
• RQ3 How can the fairness constraint be efficiently implemented in kernel methods and in linear preprocessing, and what are the implications for convexity and optimization?
• RQ4 Under what conditions does a convex surrogate (with hinge/linear losses) approximate the nonconvex fair ERM objective well?
• RQ5 How does the proposed method perform empirically against state-of-the-art fairness methods on benchmark datasets in terms of accuracy and fairness (DEO)?

Key findings

• FERM achieves competitive accuracy with substantially reduced DEO compared to baselines on several datasets.
• Theoretical results (Theorem 1) show that with appropriate ε̂, the FERM solution is consistent in risk and fairness as sample size grows.
• A convex surrogate (hinge loss) provides a tractable optimization while preserving close adherence to EO under a verifiable condition (Δ̂).
• In kernelized settings, the fairness constraint becomes an orthogonality constraint with ε=0, enabling integration with standard SVM solvers; in linear settings it corresponds to a data preprocessing step.
• Empirical results indicate Fair ERM improves fairness on four of five datasets and remains competitive on the fifth, often outperforming state-of-the-art methods like Hardt and Zafar under similar accuracy regimes.
• The approach generalizes to linear models such as Lasso, preserving sparsity and fairness when combined with the proposed preprocessing.