[Paper Review] Statistical, Robustness, and Computational Guarantees for Sliced Wasserstein Distances

Sloan Nietert, Ritwik Sadhu+2 more|arXiv (Cornell University)|Oct 17, 2022

Statistical Methods and Inference4 citations

TL;DR

This paper provides a comprehensive theoretical analysis of sliced Wasserstein distances, establishing fast empirical convergence rates under log-concave distributions, dimension-free robustness guarantees equivalent to robust mean estimation, and computational complexity bounds for Monte Carlo and subgradient methods. It demonstrates that higher dimensions can accelerate numerical integration error in average-sliced estimation and proves an O(ε⁻⁴) complexity bound for max-sliced optimization.

ABSTRACT

Sliced Wasserstein distances preserve properties of classic Wasserstein distances while being more scalable for computation and estimation in high dimensions. The goal of this work is to quantify this scalability from three key aspects: (i) empirical convergence rates; (ii) robustness to data contamination; and (iii) efficient computational methods. For empirical convergence, we derive fast rates with explicit dependence of constants on dimension, subject to log-concavity of the population distributions. For robustness, we characterize minimax optimal, dimension-free robust estimation risks, and show an equivalence between robust sliced 1-Wasserstein estimation and robust mean estimation. This enables lifting statistical and algorithmic guarantees available for the latter to the sliced 1-Wasserstein setting. Moving on to computational aspects, we analyze the Monte Carlo estimator for the average-sliced distance, demonstrating that larger dimension can result in faster convergence of the numerical integration error. For the max-sliced distance, we focus on a subgradient-based local optimization algorithm that is frequently used in practice, albeit without formal guarantees, and establish an $O(ε^{-4})$ computational complexity bound for it. Our theory is validated by numerical experiments, which altogether provide a comprehensive quantitative account of the scalability question.

Motivation & Objective

• To quantify the scalability of sliced Wasserstein distances across statistical, robustness, and computational dimensions.
• To derive fast, explicit convergence rates for empirical estimation of average- and max-sliced Wasserstein distances, with dependence on dimension.
• To characterize robust estimation risks under data contamination and establish equivalence to robust mean estimation.
• To analyze computational efficiency of Monte Carlo integration for average-sliced distances and subgradient methods for max-sliced distances.
• To validate theoretical findings with numerical experiments on synthetic and real-world data, including GAN training under contamination.

Proposed method

• Derives fast empirical convergence rates for sliced Wasserstein distances under log-concave population distributions, with explicit dependence of constants on dimension.
• Establishes equivalence between robust sliced 1-Wasserstein estimation and robust mean estimation, enabling transfer of known guarantees.
• Analyzes Monte Carlo integration error for average-sliced distances, showing that higher dimension can lead to faster convergence due to variance decay.
• Provides an O(ε⁻⁴) computational complexity bound for a subgradient-based local optimization algorithm used in max-sliced distance estimation.
• Employs concentration inequalities and metric entropy arguments to derive high-probability bounds on estimation error and robustness risk.
• Validates theoretical results through numerical experiments on Gaussian mixtures, barycenter computation, and GAN training under data contamination.

Experimental results

Research questions

• RQ1 What are the explicit empirical convergence rates of sliced Wasserstein distances, and how do they depend on dimension under log-concave distributions?
• RQ2 How does data contamination affect the estimation of sliced Wasserstein distances, and what is the minimax optimal robust risk?
• RQ3 Can the robustness of sliced Wasserstein distances be characterized in a dimension-free manner, and how does it relate to robust mean estimation?
• RQ4 How does the computational error of Monte Carlo integration for average-sliced distances scale with dimension?
• RQ5 What is the computational complexity of subgradient-based optimization for max-sliced Wasserstein distances, and can it be bounded?

Key findings

• Empirical convergence rates for average-sliced Wasserstein distances are established with explicit dependence on dimension, under log-concave distributions, achieving near-parametric rates.
• Robust estimation of sliced 1-Wasserstein distances is dimension-free and minimax optimal, with risk equivalent to that of robust mean estimation.
• The Monte Carlo estimator for average-sliced distances exhibits faster numerical integration error convergence in higher dimensions due to reduced variance of the projection distance function.
• For max-sliced Wasserstein distance, the subgradient-based local optimization algorithm has a computational complexity bound of O(ε⁻⁴) for ε-accuracy.
• Numerical experiments confirm theoretical predictions, showing improved convergence in high dimensions and robustness under contamination in GAN training.
• Theoretical guarantees for robustness and computation are validated in practical settings, including generative modeling with contaminated MNIST data.