[Paper Review] Demographic Parity: Mitigating Biases in Real-World Data

Orestis Loukas, Ho‐Ryun Chung|arXiv (Cornell University)|Sep 27, 2023

Insurance, Mortality, Demography, Risk ManagementSocial Sciences3 citations

TL;DR

This paper proposes a model-agnostic, mathematically rigorous method called PUR (Demographic Parity, Utility, and Realism) to de-bias real-world datasets by deriving an asymptotic fair distribution that enforces demographic parity while preserving classification utility and social realism. The approach uses iterative proportional fitting to project training data onto a fair distribution, which generates synthetic data that, when used to train classifiers, eliminates both direct and indirect biases, as validated on the UCI Adult dataset with near-perfect demographic parity in predictions.

ABSTRACT

Computer-based decision systems are widely used to automate decisions in many aspects of everyday life, which include sensitive areas like hiring, loaning and even criminal sentencing. A decision pipeline heavily relies on large volumes of historical real-world data for training its models. However, historical training data often contains gender, racial or other biases which are propagated to the trained models influencing computer-based decisions. In this work, we propose a robust methodology that guarantees the removal of unwanted biases while maximally preserving classification utility. Our approach can always achieve this in a model-independent way by deriving from real-world data the asymptotic dataset that uniquely encodes demographic parity and realism. As a proof-of-principle, we deduce from public census records such an asymptotic dataset from which synthetic samples can be generated to train well-established classifiers. Benchmarking the generalization capability of these classifiers trained on our synthetic data, we confirm the absence of any explicit or implicit bias in the computer-aided decision.

Motivation & Objective

• To address direct and indirect biases in real-world data that propagate into AI decision systems, especially in high-stakes domains like hiring and criminal sentencing.
• To develop a fair machine learning framework that separates de-biasing from model training, enabling model-independent fairness guarantees.
• To preserve classification utility and social realism while enforcing demographic parity in synthetic datasets derived from real census data.
• To demonstrate that classifiers trained on synthetic data from the fair distribution achieve near-zero bias in predictions on test data.

Proposed method

• The method enforces three core constraints: demographic parity (equal outcome distribution across protected groups), classification utility (preservation of original class relationships), and social realism (faithful representation of predictor relationships).
• It uses iterative proportional fitting (IPF) to project the empirical training distribution onto a fair probability distribution that satisfies all PUR constraints.
• The fair distribution is derived by minimizing the Kullback-Leibler divergence from the original data distribution under the PUR constraints, ensuring minimal distortion.
• Synthetic datasets are generated from the fair distribution using re-sampling, preserving the size and structure of the original data.
• The approach is applied to the UCI Adult dataset, where the fair distribution is used to train Random Forest classifiers on synthetic data.
• The fairness of the resulting classifiers is evaluated using a disparity ratio metric on test data, comparing performance across protected attributes.

Figure 2: Attributable disparity over salary classes estimated by the prediction on simulated test data of information projection of train distributions under various conditions. Blue line denotes the estimate $p_{\textrm{pred}}(y|\mathbf{s}_{0})$ used as reference in 7 . The original data refers ba

Experimental results

Research questions

• RQ1 Can a model-agnostic method achieve demographic parity in real-world data without compromising classification utility or realism?
• RQ2 To what extent can indirect biases—mediated through unprotected predictors—be eliminated by modifying the data distribution alone?
• RQ3 Does training classifiers on synthetic data derived from a fair distribution preserve fairness when tested on biased real-world data?
• RQ4 How well does the PUR framework maintain demographic parity across different sensitive profiles compared to baseline de-biasing methods?

Key findings

• The disparity ratio for income above $50K across protected groups (e.g., male/white vs. other) was reduced to approximately 100% on average in synthetic data generated from the PUR projection, indicating near-perfect demographic parity.
• Classifiers trained on synthetic data from the PUR projection maintained near-zero bias in predictions on test data, with the disparity ratio fluctuating around 100% and never falling below the 80% threshold.
• The p projection (demographic parity only) failed to eliminate indirect biases, as classifiers trained on such data re-introduced discriminatory correlations in predictions due to unaddressed confounding relationships.
• The pu projection (utility and parity) showed improved fairness over p alone, but only the full PUR projection—incorporating social realism—consistently preserved fairness in predictions on biased test data.
• The method successfully generated synthetic datasets that preserved the original data's size and structure while ensuring fairness, as confirmed by both direct disparity computation and classifier evaluation.
• The approach demonstrated robustness to finite-sample fluctuations, with deviations from parity attributed only to sampling noise and not systematic bias.

Figure 3: Natural predictor $p_{\textrm{pred}}(y|\mathbf{s})$ of the pur -projection of train distribution evaluated on simulated test data for all profiles in $\mathbf{S}$ . Blue line denotes $f(y)$ as computed from the empirical distribution of the original data.