[Paper Review] Two-sample Test using Projected Wasserstein Distance: Breaking the Curse of Dimensionality.

Jie Wang, Rui Gao+1 more|arXiv (Cornell University)|Oct 22, 2020

Anomaly Detection Techniques and Applications41 references4 citations

TL;DR

This paper proposes a projected Wasserstein distance to overcome the curse of dimensionality in two-sample testing by optimizing a low-dimensional linear projection that maximizes the Wasserstein distance between projected distributions. The method achieves improved finite-sample convergence rates and maintains testing power in high dimensions, with theoretical guarantees and practical algorithms provided.

ABSTRACT

We develop a projected Wasserstein distance for the two-sample test, a fundamental problem in statistics and machine learning: given two sets of samples, to determine whether they are from the same distribution. In particular, we aim to circumvent the curse of dimensionality in Wasserstein distance: when the dimension is high, it has diminishing testing power, which is inherently due to the slow concentration property of Wasserstein metrics in the high dimension space. A key contribution is to couple optimal projection to find the low dimensional linear mapping to maximize the Wasserstein distance between projected probability distributions. We characterize the theoretical property of the finite-sample convergence rate on IPMs and present practical algorithms for computing this metric. Numerical examples validate our theoretical results.

Motivation & Objective

• Address the curse of dimensionality in Wasserstein-based two-sample tests, where performance degrades as dimension increases.
• Develop a method that maintains statistical power in high-dimensional settings by projecting data into lower-dimensional subspaces.
• Optimize the projection direction to maximize the Wasserstein distance between projected distributions.
• Establish theoretical finite-sample convergence rates for the proposed metric within the framework of integral probability metrics (IPMs).
• Provide practical algorithms for computing the projected Wasserstein distance in real-world applications.

Proposed method

• Introduce a projected Wasserstein distance by applying an optimal linear projection to high-dimensional data before computing the Wasserstein distance.
• Formulate the projection optimization as a maximization problem over the Grassmann manifold to find the direction that maximizes the Wasserstein distance between projected empirical measures.
• Leverage the theory of integral probability metrics (IPMs) to derive finite-sample convergence rates for the projected metric.
• Use a stochastic gradient-based approach to efficiently compute the optimal projection direction in practice.
• Ensure theoretical consistency by analyzing the concentration properties of the projected Wasserstein distance under mild regularity conditions.
• Integrate the projected distance into a two-sample hypothesis test with controlled Type I error and improved power in high dimensions.

Experimental results

Research questions

• RQ1 Can optimal linear projection improve the finite-sample convergence rate of the Wasserstein distance in high-dimensional two-sample testing?
• RQ2 Does maximizing the projected Wasserstein distance lead to better testing power compared to standard Wasserstein distance in high dimensions?
• RQ3 How does the proposed method perform in terms of Type I error control and power under various distributional assumptions?
• RQ4 What is the theoretical finite-sample convergence rate of the projected Wasserstein distance as a member of the IPM family?
• RQ5 Can the method be efficiently computed in practice, and how does it scale with dimension and sample size?

Key findings

• The projected Wasserstein distance achieves a faster finite-sample convergence rate than the standard Wasserstein distance in high-dimensional settings.
• Optimal projection significantly enhances testing power by mitigating the curse of dimensionality in two-sample testing.
• The method maintains valid Type I error control under the null hypothesis, ensuring statistical reliability.
• Theoretical analysis confirms that the projected metric belongs to the IPM family and inherits favorable concentration properties.
• Numerical experiments demonstrate consistent performance gains over baseline Wasserstein distance across various high-dimensional distributions.
• The proposed algorithm efficiently computes the optimal projection direction using stochastic optimization, enabling scalability to high-dimensional data.