[Paper Review] Estimation of smooth densities in Wasserstein distance

Jonathan Weed, Quentin Berthet|arXiv (Cornell University)|Feb 5, 2019

Geometric Analysis and Curvature Flows38 citations

TL;DR

This paper establishes the first minimax optimal rates for estimating smooth densities in Wasserstein distance, demonstrating that regularity of the density mitigates the curse of dimensionality. By introducing novel bounds between Wasserstein distances and Besov norms, the authors derive improved convergence rates and construct computationally efficient, discretely supported approximations.

ABSTRACT

The Wasserstein distances are a set of metrics on probability distributions supported on $\mathbb{R}^d$ with applications throughout statistics and machine learning. Often, such distances are used in the context of variational problems, in which the statistician employs in place of an unknown measure a proxy constructed on the basis of independent samples. This raises the basic question of how well measures can be approximated in Wasserstein distance. While it is known that an empirical measure comprising i.i.d. samples is rate-optimal for general measures, no improved results were known for measures possessing smooth densities. We prove the first minimax rates for estimation of smooth densities for general Wasserstein distances, thereby showing how the curse of dimensionality can be alleviated for sufficiently regular measures. We also show how to construct discretely supported measures, suitable for computational purposes, which enjoy improved rates. Our approach is based on novel bounds between the Wasserstein distances and suitable Besov norms, which may be of independent interest.

Motivation & Objective

• To address the gap in minimax rates for smooth density estimation under Wasserstein distance, particularly for regular measures.
• To show that smoothness of the density reduces the curse of dimensionality in Wasserstein estimation.
• To develop discretely supported measures that maintain improved convergence rates for computational use.
• To establish theoretical foundations via new bounds between Wasserstein distance and Besov norms.

Proposed method

• Derive new theoretical bounds linking Wasserstein distance to Besov norms, enabling tighter control over estimation error.
• Use these bounds to analyze the minimax risk for smooth densities in general Wasserstein distances.
• Construct discretely supported approximations of the true measure using quantization or sampling strategies that preserve convergence rates.
• Leverage the regularity of the density to derive improved rates compared to general measures.
• Apply variational inference principles to replace unknown measures with empirical proxies, analyzing their convergence in Wasserstein distance.
• Use tools from functional analysis and optimal transport to characterize the trade-off between smoothness and approximation error.

Experimental results

Research questions

• RQ1 Can improved minimax rates be achieved for smooth densities in Wasserstein distance compared to general measures?
• RQ2 How does the smoothness of a density influence the convergence rate of its Wasserstein approximation?
• RQ3 What is the optimal trade-off between approximation accuracy and computational efficiency in discretely supported measures?
• RQ4 Can novel bounds between Wasserstein distance and Besov norms lead to tighter estimation error control?
• RQ5 To what extent does regularity of the density alleviate the curse of dimensionality in Wasserstein estimation?

Key findings

• The paper establishes the first minimax optimal rates for estimating smooth densities in general Wasserstein distances, showing improved convergence over general measures.
• Smoothness of the density leads to a significant reduction in the effective dimensionality, mitigating the curse of dimensionality.
• Novel bounds between Wasserstein distance and Besov norms are derived, which are of independent theoretical interest.
• Discretely supported measures can be constructed that achieve the same improved rates as continuous approximations, enabling practical computation.
• The rates depend on the smoothness of the density and the dimension, with faster convergence for higher regularity.
• The results demonstrate that empirical measures are rate-optimal for general measures, but improved rates are possible under smoothness assumptions.