[Paper Review] Towards Optimal Running Times for Optimal Transport

José Blanchet, Arun Jambulapati+2 more|arXiv (Cornell University)|Oct 17, 2018

Complexity and Algorithms in Graphs4 citations

TL;DR

This paper presents two new algorithms for approximating the optimal transport distance between discrete probability distributions in $Ϫ(n^2 \|C\|_{\infty}/\epsilon)$ time, improving upon prior bounds. By reducing optimal transport to packing linear programs and matrix balancing, the authors achieve faster, nearly-linear time complexity and show that further improvements would imply breakthroughs in bipartite matching algorithms.

ABSTRACT

In this work, we provide faster algorithms for approximating the optimal transport distance, e.g. earth mover's distance, between two discrete probability distributions $μ, ν\in Δ^n$. Given a cost function $C : [n] imes [n] o \mathbb{R}_{\geq 0}$ where $C(i,j) \leq 1$ quantifies the penalty of transporting a unit of mass from $i$ to $j$, we show how to compute a coupling $X$ between $r$ and $c$ in time $\widetilde{O}\left(n^2 /ε ight)$ whose expected transportation cost is within an additive $ε$ of optimal. This improves upon the previously best known running time for this problem of $\widetilde{O}\left( ext{min}\left\{ n^{9/4}/ε, n^2/ε^2 ight\} ight)$. We achieve our results by providing reductions from optimal transport to canonical optimization problems for which recent algorithmic efforts have provided nearly-linear time algorithms. Leveraging nearly linear time algorithms for solving packing linear programs and for solving the matrix balancing problem, we obtain two separate proofs of our stated running time. Further, one of our algorithms is easily parallelized and can be implemented with depth $\widetilde{O}(1/ε)$. Moreover, we show that further algorithmic improvements to our result would be surprising in the sense that any improvement would yield an $o(n^{2.5})$ algorithm for extit{maximum cardinality bipartite matching}, for which currently the only known algorithms for achieving such a result are based on fast-matrix multiplication.

Motivation & Objective

• Address the need for faster algorithms to compute optimal transport distances in machine learning and statistics applications.
• Improve upon the best-known running time for computing $ε$-optimal solutions to the discrete optimal transport problem.
• Provide a reduction-based approach that leverages nearly-linear time algorithms for canonical problems in theoretical computer science.
• Demonstrate that further improvements to the proposed running time would imply a $o(n^{2.5})$ algorithm for maximum cardinality bipartite matching.
• Achieve both serial and parallel algorithmic improvements, including a depth of $\widetilde{O}(\|C\|_{\infty}/\epsilon)$ for the parallel variant.

Proposed method

• Reduce the optimal transport problem to a packing linear program (PLP) and use nearly-linear time solvers for PLPs to obtain one algorithm.
• Reduce the optimal transport problem to the matrix balancing problem and use nearly-linear time matrix balancing solvers for a second algorithm.
• Use a black-box reduction framework to transform the optimal transport problem into a form solvable by state-of-the-art iterative solvers.
• Apply randomized and deterministic rounding techniques to convert fractional solutions into integral matchings with bounded error.
• Leverage known nearly-linear time algorithms for solving packing LPs and matrix balancing to achieve the final complexity bounds.
• Use a hardness reduction to show that any improvement beyond $\widetilde{O}(n^2 \|C\|_{\infty}/\epsilon)$ would imply a $o(n^{2.5})$ algorithm for maximum cardinality bipartite matching.

Experimental results

Research questions

• RQ1 Can the running time for computing $\epsilon$-optimal optimal transport solutions be improved beyond the current best-known $\widetilde{O}(\min\{n^{9/4}\sqrt{\|C\|_{\infty}}/\epsilon, n^2\|C\|_{\infty}/\epsilon^2\})$?
• RQ2 Is it possible to achieve a nearly-linear time algorithm for optimal transport by reducing it to canonical problems like packing LPs or matrix balancing?
• RQ3 What is the relationship between improvements in optimal transport complexity and breakthroughs in bipartite matching algorithms?
• RQ4 Can the proposed algorithms be efficiently parallelized, and what is the achievable parallel depth?
• RQ5 How do the reductions to packing LPs and matrix balancing compare in terms of complexity and practicality?

Key findings

• The paper achieves a running time of $\widetilde{O}(n^2 \|C\|_{\infty}/\epsilon)$ for computing $\epsilon$-optimal transport couplings, improving upon the previous best of $\widetilde{O}(\min\{n^{9/4}\sqrt{\|C\|_{\infty}}/\epsilon, n^2\|C\|_{\infty}/\epsilon^2\})$.
• One of the proposed algorithms achieves a parallel depth of $\widetilde{O}(\|C\|_{\infty}/\epsilon)$, improving the previous best parallel depth by a factor of $1/\epsilon$.
• The authors show that any improvement to the $\widetilde{O}(n^2 \|C\|_{\infty}/\epsilon)$ running time would yield a $o(n^{2.5})$ algorithm for maximum cardinality bipartite matching.
• The hardness result implies that further improvements would require a breakthrough in algorithmic graph theory, as no such algorithm is known without fast matrix multiplication.
• The reductions to packing LPs and matrix balancing are black-box and allow the use of state-of-the-art nearly-linear time solvers, enabling the new complexity bounds.
• The results are robust to general cost functions without metric assumptions, making them applicable to a wide range of machine learning and statistical applications.