[Paper Review] Limit distribution theory for smooth $p$-Wasserstein distances

Ziv Goldfeld, Kengo Kato+2 more|arXiv (Cornell University)|Mar 1, 2022

Geometric Analysis and Curvature Flows4 citations

TL;DR

This paper establishes a comprehensive limit distribution theory for the empirical smooth $p$-Wasserstein distance using functional delta method and dual Sobolev space embeddings, proving weak convergence of the smooth empirical process and consistency of the nonparametric bootstrap. It further establishes asymptotic normality of minimum distance estimators in generative modeling under quadratic cost.

ABSTRACT

The Wasserstein distance is a metric on a space of probability measures that has seen a surge of applications in statistics, machine learning, and applied mathematics. However, statistical aspects of Wasserstein distances are bottlenecked by the curse of dimensionality, whereby the number of data points needed to accurately estimate them grows exponentially with dimension. Gaussian smoothing was recently introduced as a means to alleviate the curse of dimensionality, giving rise to a parametric convergence rate in any dimension, while preserving the Wasserstein metric and topological structure. To facilitate valid statistical inference, in this work, we develop a comprehensive limit distribution theory for the empirical smooth Wasserstein distance. The limit distribution results leverage the functional delta method after embedding the domain of the Wasserstein distance into a certain dual Sobolev space, characterizing its Hadamard directional derivative for the dual Sobolev norm, and establishing weak convergence of the smooth empirical process in the dual space. To estimate the distributional limits, we also establish consistency of the nonparametric bootstrap. Finally, we use the limit distribution theory to study applications to generative modeling via minimum distance estimation with the smooth Wasserstein distance, showing asymptotic normality of optimal solutions for the quadratic cost.

Motivation & Objective

• To address the curse of dimensionality in statistical inference for Wasserstein distances by leveraging Gaussian smoothing to achieve parametric convergence rates.
• To develop a rigorous limit distribution theory for the empirical smooth $p$-Wasserstein distance when $p > 1$, where prior results were limited to $p=1$.
• To establish weak convergence of the smooth empirical process in a dual Sobolev space and characterize the Hadamard directional derivative of the smooth Wasserstein distance.
• To validate statistical inference by proving consistency of the nonparametric bootstrap for the smooth Wasserstein distance.
• To apply the limit theory to minimum distance estimation in generative modeling, showing asymptotic normality of optimal parameters under quadratic cost.

Proposed method

• Embed the domain of the smooth $p$-Wasserstein distance into a dual Sobolev space $\dot{H}^{-1,p}(\mu * \gamma_\sigma)$ to enable functional analytic tools.
• Characterize the Hadamard directional derivative of the smooth Wasserstein distance in the dual Sobolev norm to enable application of the functional delta method.
• Establish weak convergence of the smooth empirical process $\tilde{\mathbb{G}}_n^{(\sigma)}$ in the dual Sobolev space under mild moment conditions.
• Apply the functional delta method to derive the asymptotic distribution of $\sqrt{n} \, \mathsf{W}_p^{(\sigma)}(\hat{\mu}_n, \mu)$ as the supremum of a tight Gaussian process.
• Prove consistency of the nonparametric bootstrap for estimating the distributional limits of the smooth Wasserstein distance.
• Use the limit theory to analyze minimum distance estimation in generative modeling, showing asymptotic normality of the optimal parameter under quadratic cost.

Experimental results

Research questions

• RQ1 Can a limit distribution theory be developed for the empirical smooth $p$-Wasserstein distance when $p > 1$, overcoming the lack of an integral probability metric structure?
• RQ2 Does the smooth $p$-Wasserstein distance retain the metric and topological properties of the original $p$-Wasserstein distance while achieving parametric convergence rates?
• RQ3 Can the functional delta method be applied to the smooth Wasserstein distance via dual Sobolev space embeddings to derive weak convergence of the empirical process?
• RQ4 Is the nonparametric bootstrap consistent for estimating the asymptotic distribution of the smooth $p$-Wasserstein distance?
• RQ5 Does the limit distribution theory enable asymptotic normality of minimum distance estimators in generative modeling using the smooth $p$-Wasserstein distance?

Key findings

• The scaled empirical smooth $p$-Wasserstein distance $\sqrt{n} \, \mathsf{W}_p^{(\sigma)}(\hat{\mu}_n, \mu)$ converges in distribution to the supremum of a tight Gaussian process in every dimension $d$ under mild moment conditions.
• The limit distribution is characterized through weak convergence of the smooth empirical process $\tilde{\mathbb{G}}_n^{(\sigma)}$ in the dual Sobolev space $\dot{H}^{-1,p}(\mu * \gamma_\sigma)$.
• The Hadamard directional derivative of the smooth Wasserstein distance is fully characterized in the dual Sobolev norm, enabling application of the functional delta method.
• The nonparametric bootstrap is consistent for estimating the asymptotic distribution of the smooth $p$-Wasserstein distance, supporting valid inference.
• In generative modeling via minimum distance estimation with quadratic cost, the optimal parameter estimator is asymptotically normal, with convergence rate $n^{-1/2}$.
• The theory confirms that Gaussian smoothing preserves the metric and topological structure of the original $p$-Wasserstein distance while eliminating the curse of dimensionality.