[Paper Review] Parametric Fairness with Statistical Guarantees

François Hu, Philipp Ratz+1 more|arXiv (Cornell University)|Oct 31, 2023

Ethics and Social Impacts of AI4 citations

TL;DR

This paper introduces parametric fair solutions using Wasserstein barycenters to enforce demographic parity while incorporating domain-specific distributional knowledge, ensuring statistical guarantees, stability, and robustness to measurement errors and latent sensitive variables. The method achieves improved fairness with minimal predictive accuracy loss, especially under data corruption or hidden biases.

ABSTRACT

Algorithmic fairness has gained prominence due to societal and regulatory concerns about biases in Machine Learning models. Common group fairness metrics like Equalized Odds for classification or Demographic Parity for both classification and regression are widely used and a host of computationally advantageous post-processing methods have been developed around them. However, these metrics often limit users from incorporating domain knowledge. Despite meeting traditional fairness criteria, they can obscure issues related to intersectional fairness and even replicate unwanted intra-group biases in the resulting fair solution. To avoid this narrow perspective, we extend the concept of Demographic Parity to incorporate distributional properties in the predictions, allowing expert knowledge to be used in the fair solution. We illustrate the use of this new metric through a practical example of wages, and develop a parametric method that efficiently addresses practical challenges like limited training data and constraints on total spending, offering a robust solution for real-life applications.

Motivation & Objective

• To address the limitations of agnostic post-processing fairness methods that discard domain knowledge and fail to handle intersectional or latent biases.
• To ensure prediction stability and smooth transition from unfair to fair models, especially in real-world applications like wage setting with fixed budgets.
• To develop a parametric framework that embeds expert knowledge into fair prediction distributions, improving fairness beyond standard Demographic Parity.
• To provide theoretical guarantees on fairness, risk, and budget constraints in parametric fair solutions.
• To evaluate robustness under data errors and latent sensitive variables, where standard nonparametric methods fail.

Proposed method

• Uses Wasserstein barycenters to compute fair predictions that minimize distance to base models while satisfying fairness constraints.
• Imposes parametric shape constraints (e.g., Gaussian, Gumbel, Beta) on the fair output distribution to embed domain knowledge.
• Applies closed-form optimal transport plans from Chzhen et al. (2020c) for computational efficiency and seamless integration.
• Employs a plug-in estimator using estimated parametric parameters from training data to generate fair predictions.
• Introduces a risk-fairness trade-off measure based on Wasserstein distance to enable multi-period fairness transitions.
• Uses parametric forms to correct for measurement errors in training data, avoiding reliance on input features or transfer learning.

Figure 1: Base predictors are shown in blue and orange, while the optimal fair predictor is in green. In this example, integrating fairness considerations with domain knowledge effectively mitigates intersectional fairness issues. Here, $g^{*}$ corresponds to the Bayes rule, while $g^{*(\text{fair})

Experimental results

Research questions

• RQ1 Can parametric constraints improve fairness while preserving predictive accuracy in the presence of latent sensitive variables?
• RQ2 How does incorporating domain knowledge via parametric shape constraints affect fairness and stability in post-processing fairness methods?
• RQ3 To what extent can parametric fair models reduce bias when training data contains measurement errors?
• RQ4 Can parametric fairness methods ensure a smooth transition from unfair to fair regimes with minimal disruption to overall allocations?
• RQ5 How do different parametric families (e.g., Gaussian, Beta, Gumbel) affect the fairness-accuracy trade-off in real-world datasets?

Key findings

• The parametric fair method reduced unfairness on latent sensitive variables by up to 25% compared to nonparametric baselines, even when the latent variable was unobserved during training.
• In the presence of 75% measurement error in training data, the parametric method reduced mean squared error (MSE) by 16.5% compared to the uncorrected baseline on the ACSIncome dataset.
• For the folktables ACSPublicCoverage task, the parametric estimator achieved a 0.536 F1 score with ε=0.75, outperforming the nonparametric method (0.535) while maintaining fairness.
• The method preserved overall prediction stability, keeping average scores consistent across fairness transitions, crucial for budget-constrained applications like wage adjustments.
• Theoretical guarantees show convergence of parametric estimators to optimal values and bounded total budget requirements relative to the optimal case.
• The parametric approach maintained or slightly reduced predictive accuracy (e.g., MSE increase of ~0.01 in some cases) while significantly improving fairness under latent or corrupted data.

Figure 2: Approximate fairness between two gaussian distributions and their barycenter