[Paper Review] A Central Limit Theorem for $L_p$ transportation cost with applications to Fairness Assessment in Machine Learning

Eustasio del Barrio, Paula Gordaliza+1 more|arXiv (Cornell University)|Jul 18, 2018

Adversarial Robustness in Machine Learning18 references3 citations

TL;DR

This paper establishes a Central Limit Theorem for the $L_p$-Wasserstein distance between empirical distributions on the real line under minimal moment and smoothness assumptions. It enables asymptotically valid two-sample tests for distribution similarity and introduces a novel fairness assessment criterion in machine learning by quantifying the Wasserstein distance between classifier outputs across protected subgroups.

ABSTRACT

We provide a Central Limit Theorem for the Monge-Kantorovich distance between two empirical distributions with size $n$ and $m$, $W_p(P_n,Q_m)$ for $p>1$ for observations on the real line, using a minimal amount of assumptions. We provide an estimate of the asymptotic variance which enables to build a two sample test to assess the similarity between two distributions. This test is then used to provide a new criterion to assess the notion of fairness of a classification algorithm.

Motivation & Objective

• To establish a Central Limit Theorem (CLT) for the $L_p$-Wasserstein distance $W_p(P_n, Q_m)$ between two empirical distributions with minimal assumptions on the underlying distributions.
• To derive an estimate of the asymptotic variance of $W_p(P_n, Q_m)$ to enable the construction of a two-sample hypothesis test for distribution similarity.
• To apply the theoretical CLT to develop a new, distribution-free criterion for assessing fairness in machine learning models by measuring the Wasserstein distance between classifier outputs across protected subgroups.
• To provide a rigorous statistical foundation for fairness evaluation that quantifies bias through optimal transport distances rather than relying on independence or conditional independence assumptions.

Proposed method

• Derives the asymptotic distribution of the $L_p$-Wasserstein distance $W_p(P_n, Q_m)$ between empirical measures $P_n$ and $Q_m$ with sample sizes $n$ and $m$, under minimal moment and smoothness conditions on the cumulative distribution functions.
• Uses a Taylor expansion of the $p$-th power of the Wasserstein distance and applies weak convergence results for empirical quantile processes to derive the limiting normal distribution.
• Establishes convergence in distribution of the normalized $W_p^p$ distance to a normal law, with an explicit expression for the asymptotic variance derived from the inverse cumulative distribution functions and their differences.
• Applies the CLT to construct a two-sample test for the null hypothesis $\mathcal{W}_p(P,Q) \geq \Delta_0$ versus $\mathcal{W}_p(P,Q) < \Delta_0$, using the estimated asymptotic variance.
• Proposes a fairness metric in machine learning based on the Wasserstein distance between the conditional distributions of model predictions given a protected attribute $S=0$ and $S=1$.
• Uses the theoretical CLT to justify inference on the fairness metric, enabling p-values and confidence intervals for assessing statistical significance of observed disparities.

Experimental results

Research questions

• RQ1 Under what minimal regularity conditions does the $L_p$-Wasserstein distance between two empirical distributions on the real line satisfy a Central Limit Theorem?
• RQ2 Can the asymptotic variance of the $W_p(P_n, Q_m)$ statistic be consistently estimated under weak moment and smoothness assumptions?
• RQ3 How can the CLT for $W_p(P_n, Q_m)$ be leveraged to construct a valid two-sample test for distributional similarity?
• RQ4 To what extent can the Wasserstein distance between classifier outputs across protected subgroups serve as a statistically interpretable fairness metric in machine learning?
• RQ5 What is the asymptotic behavior of the expected $W_p^p$ distance between empirical and true distributions, and how does it relate to the limiting normal distribution?

Key findings

• The paper establishes that $\sqrt{n}(W_p^p(F_n, G) - W_p^p(F, G))$ converges in distribution to a normal random variable with mean zero and a well-defined asymptotic variance under minimal assumptions.
• The asymptotic variance of the $W_p^p$ distance is derived explicitly in terms of the inverse cumulative distribution functions of the underlying distributions and their differences.
• The CLT holds under only $p$-th moment conditions and mild smoothness of the cumulative distribution functions, improving upon prior results requiring higher-order moments.
• The limiting distribution of $W_p(P_n, Q_m)$ is asymptotically normal, enabling the construction of confidence intervals and hypothesis tests for distributional similarity.
• The expected value of $W_p^p(F_n, G)$ converges to $W_p^p(F, G)$ at rate $O(1/\sqrt{n})$, and the centered sequence converges in distribution to a normal law.
• The proposed fairness assessment framework quantifies bias via the Wasserstein distance between model outputs for protected and unprotected groups, with statistical inference enabled by the derived CLT.