[Paper Review] Tree-Sliced Variants of Wasserstein Distances

Tam Le, Makoto Yamada+2 more|arXiv (Cornell University)|Feb 1, 2019

3D Shape Modeling and AnalysisEngineering59 references3 citations

TL;DR

This paper introduces tree-sliced Wasserstein distances, a novel variant of optimal transport that leverages random tree metrics to enable fast, closed-form computation of Wasserstein distances. By averaging tree-Wasserstein distances over multiple adaptive tree structures in low- or high-dimensional spaces, the method achieves computational efficiency while preserving statistical properties, including negative definiteness, which enables the construction of a positive definite kernel for downstream machine learning tasks with strong empirical performance.

ABSTRACT

Optimal transport (\OT) theory defines a powerful set of tools to compare probability distributions. \OT~suffers however from a few drawbacks, computational and statistical, which have encouraged the proposal of several regularized variants of OT in the recent literature, one of the most notable being the extit{sliced} formulation, which exploits the closed-form formula between univariate distributions by projecting high-dimensional measures onto random lines. We consider in this work a more general family of ground metrics, namely extit{tree metrics}, which also yield fast closed-form computations and negative definite, and of which the sliced-Wasserstein distance is a particular case (the tree is a chain). We propose the tree-sliced Wasserstein distance, computed by averaging the Wasserstein distance between these measures using random tree metrics, built adaptively in either low or high-dimensional spaces. Exploiting the negative definiteness of that distance, we also propose a positive definite kernel, and test it against other baselines on a few benchmark tasks.

Motivation & Objective

• To address the high computational cost of optimal transport (OT) in high-dimensional spaces by introducing a fast, scalable alternative.
• To generalize the sliced-Wasserstein distance by replacing linear projections with tree-structured metrics, enabling more flexible and adaptive geometry.
• To exploit the negative definiteness of tree-Wasserstein distances to construct a positive definite kernel for use in kernel-based machine learning methods.
• To evaluate the proposed method on benchmark tasks, including word embedding-based document classification and topological data analysis, to demonstrate empirical superiority.

Proposed method

• Proposes tree-sliced Wasserstein distance by averaging tree-Wasserstein (TW) distances computed over multiple randomly sampled tree metrics built from spatial data structures.
• Constructs tree metrics using hierarchical clustering or space-filling curves (e.g., Hilbert curves) to adaptively partition data in low- or high-dimensional spaces.
• Uses the closed-form solution for Wasserstein distance on tree metrics, which enables efficient computation without solving full OT problems.
• Leverages the negative definiteness of the tree-Wasserstein distance to derive a positive definite kernel via the exponential transform, enabling kernel methods.
• Employs a recursive partitioning algorithm (e.g., Partition_HC) to build tree structures from data points or their spatial regions, discarding empty subregions.
• Applies the method to both discrete and continuous distributions, with theoretical grounding in optimal transport duality and metric geometry.

Experimental results

Research questions

• RQ1 Can tree metrics serve as a computationally efficient alternative to standard ground metrics in optimal transport?
• RQ2 Does averaging tree-Wasserstein distances over multiple random trees yield a stable and informative divergence between probability measures?
• RQ3 Can the negative definiteness of tree-Wasserstein distances be exploited to construct a positive definite kernel for kernel-based learning?
• RQ4 How does the tree-sliced Wasserstein distance compare to existing baselines like sliced-Wasserstein and standard OT in downstream machine learning tasks?

Key findings

• The tree-sliced Wasserstein distance achieves significant speedups over standard OT while maintaining competitive performance on benchmark datasets.
• The method outperforms baseline approaches, including sliced-Wasserstein and standard OT, in word embedding-based document classification tasks.
• The proposed positive definite kernel based on tree-sliced distances achieves state-of-the-art results in topological data analysis, particularly in persistence diagram classification.
• The tree-sliced approach mitigates cluster sensitivity and quantization issues common in hierarchical clustering and partitioning by averaging over multiple random tree structures.
• Theoretical analysis confirms that tree-Wasserstein distances are negative definite, enabling the construction of a valid positive definite kernel via exponential transformation.