[Paper Review] Subspace Robust Wasserstein Distances

François-Pierre Paty, Marco Cuturi|arXiv (Cornell University)|Jan 25, 2019

Point processes and geometric inequalitiesMathematics55 citations

TL;DR

This paper introduces Subspace Robust Wasserstein Distances (SRW) as a robust variant of optimal transport, using a k-dimensional subspace to define a max-min (or min-max) transport cost, with a tight convex relaxation and entropic-regularized algorithms for computation.

ABSTRACT

Making sense of Wasserstein distances between discrete measures in high-dimensional settings remains a challenge. Recent work has advocated a two-step approach to improve robustness and facilitate the computation of optimal transport, using for instance projections on random real lines, or a preliminary quantization of the measures to reduce the size of their support. We propose in this work a "max-min" robust variant of the Wasserstein distance by considering the maximal possible distance that can be realized between two measures, assuming they can be projected orthogonally on a lower $k$-dimensional subspace. Alternatively, we show that the corresponding "min-max" OT problem has a tight convex relaxation which can be cast as that of finding an optimal transport plan with a low transportation cost, where the cost is alternatively defined as the sum of the $k$ largest eigenvalues of the second order moment matrix of the displacements (or matchings) corresponding to that plan (the usual OT definition only considers the trace of that matrix). We show that both quantities inherit several favorable properties from the OT geometry. We propose two algorithms to compute the latter formulation using entropic regularization, and illustrate the interest of this approach empirically.

Motivation & Objective

• Motivate instability of standard OT in high dimensions and the need for robust metrics.
• Introduce projection and subspace robustness concepts for Wasserstein distances.
• Derive a convex relaxation that reduces to a low-rank eigenvalue objective.
• Provide algorithms to compute SRW via entropic regularization and saddle-point optimization.
• Demonstrate robustness and practical performance on synthetic and real data.

Proposed method

• Define k-dimensional projection robust Wasserstein distance PRW and k-dimensional subspace robust Wasserstein distance SRW.
• Show that SRW can be written as a min over transport plans of the sum of the k largest eigenvalues of the displacement second-order moment V_pi.
• Prove a tight convex relaxation (Theorem 1) equating SRW^2 to a max over k-dim Omega of a Wasserstein cost with Mahalanobis distance d_Omega.
• Develop computation strategies: projected supergradient method and Frank-Wolfe with entropy regularization to optimize Omega and the transport plan.
• Discuss initialization and stopping criteria, including warm-starts and iterative schemes for k from d down to 1.

Experimental results

Research questions

• RQ1 How can Wasserstein distances be made robust to high-dimensional perturbations via subspace projections?
• RQ2 Can we reformulate the max-min projection problem into a tractable convex relaxation?
• RQ3 What is the relationship between SRW and standard OT in terms of geometry and distances?
• RQ4 How can we efficiently compute SRW using entropic regularization and saddle-point optimization?
• RQ5 Do SRW distances exhibit robustness to noise and preserve geodesic structure?

Key findings

• SRW inherits key OT properties and is equivalent to Wasserstein up to dimension-dependent constants, with explicit bounds relating S_k to W.
• SRW defines a geodesic, constant-speed interpolation in the SRW space, mirroring OT's geodesic structure.
• The sum of the top-k eigenvalues of the displacement covariance matrix governs SRW, enabling a convex relaxation for computation.
• Two practical algorithms are proposed: a projected supergradient method and a Frank-Wolfe method with entropy regularization, both leveraging inner OT solves.
• Empirical results on fragmented hypercubes, multi-variate Gaussian-like settings, and real movie-script data demonstrate SRW’s robustness to noise and its favorable behavior under dimensionality changes.