[Paper Review] Probably Approximately Metric-Fair Learning

Guy N. Rothblum, Gal Yona|arXiv (Cornell University)|Mar 8, 2018

Machine Learning and Algorithms23 references43 citations

TL;DR

The paper introduces a relaxed PACF framework for metric-fair learning, proving generalization guarantees and providing polynomial-time PACF learners for linear and logistic predictors, while showing hardness for perfect metric-fairness.

ABSTRACT

The seminal work of Dwork {\em et al.} [ITCS 2012] introduced a metric-based notion of individual fairness. Given a task-specific similarity metric, their notion required that every pair of similar individuals should be treated similarly. In the context of machine learning, however, individual fairness does not generalize from a training set to the underlying population. We show that this can lead to computational intractability even for simple fair-learning tasks. With this motivation in mind, we introduce and study a relaxed notion of {\em approximate metric-fairness}: for a random pair of individuals sampled from the population, with all but a small probability of error, if they are similar then they should be treated similarly. We formalize the goal of achieving approximate metric-fairness simultaneously with best-possible accuracy as Probably Approximately Correct and Fair (PACF) Learning. We show that approximate metric-fairness {\em does} generalize, and leverage these generalization guarantees to construct polynomial-time PACF learning algorithms for the classes of linear and logistic predictors.

Motivation & Objective

• Motivate relaxing perfect metric-fairness to approximate metric-fairness to enable generalization across the population.
• Develop PACF learning as a framework balancing fairness and best-possible accuracy.
• Establish generalization bounds for approximate metric-fairness using Rademacher complexity.
• Provide efficient (polynomial-time) PACF algorithms for linear and logistic predictors.
• Contrast with hardness results for perfect metric-fairness to justify the relaxed approach.

Proposed method

• Define approximate metric-fairness (MF) with parameters (α, γ) and failure probability δ.
• Formulate Probably Approximately Correct and Fair (PACF) learning combining MF with accuracy guarantees relative to the best approximately MF predictor.
• Prove fairness-generalization bounds using Rademacher complexity to ensure training MF implies population MF.
• Design polynomial-time relaxed-PACF learners for linear predictors (H_lin) via bounded empirical MF loss and convex MF-violation constraints.
• Extend to logistic predictors (H_{φ,L}) using kernel-based improper learning and convexification via RHKS with a polynomial kernel; achieve relaxed PACF results.
• Provide information-theoretic and computational hardness discussions contrasting perfect vs. approximate MF.

Experimental results

Research questions

• RQ1 Can approximate metric-fairness generalize from finite samples to the underlying population?
• RQ2 What are the sample complexity and generalization guarantees for PACF learning under MF constraints?
• RQ3 Can we design efficient (polynomial-time) PACF learners for linear and logistic predictors under MF?
• RQ4 How does relaxing MF (α, γ) affect accuracy guarantees relative to the best approximately MF predictor?
• RQ5 What are the computational barriers of perfect metric-fairness, and does relaxing MF bypass these barriers?

Key findings

• Approximate MF generalizes: empirical MF on a sample implies MF on the underlying distribution with high probability (Theorem 1.3).
• Information-theoretic strong PACF learnability is achievable with sample complexity similar to standard PAC learning for MF classes.
• Polynomial-time relaxed-PACF learners exist for linear predictors (H_lin) with poly-time complexity in 1/ε parameters and sample size.
• Polynomial-time relaxed-PACF learners exist for logistic predictors (H_{φ,L}) with exponential dependence on the Lipschitz parameter L due to kernel-based embedding.
• For perfect metric-fairness, some tasks remain hard even with simple predictors, motivating the shift to approximate MF (PACF).
• H_lin achieves relaxed PACF learnability; H_{φ,L} is learnable in the relaxed sense with kernelization; both maintain competitive accuracy under MF constraints.