[Paper Review] A Graph Theoretic Additive Approximation of Optimal Transport

Nathaniel Lahn, Deepika Mulchandani+1 more|arXiv (Cornell University)|May 28, 2019

Complexity and Algorithms in Graphs28 references4 citations

TL;DR

This paper presents a novel graph-theoretic algorithm that computes a δ-close additive approximation to the optimal transport problem in O(n²C/δ + nC²/δ²) time by adapting Gabow and Tarjan's cost-scaling framework. The method uses only Dijkstra-type and depth-first searches over a residual graph, achieving competitive performance with Sinkhorn on MNIST data while remaining stable at small δ values.

ABSTRACT

Transportation cost is an attractive similarity measure between probability distributions due to its many useful theoretical properties. However, solving optimal transport exactly can be prohibitively expensive. Therefore, there has been significant effort towards the design of scalable approximation algorithms. Previous combinatorial results [Sharathkumar, Agarwal STOC '12, Agarwal, Sharathkumar STOC '14] have focused primarily on the design of near-linear time multiplicative approximation algorithms. There has also been an effort to design approximate solutions with additive errors [Cuturi NIPS '13, Altschuler \etal\ NIPS '17, Dvurechensky \etal\, ICML '18, Quanrud, SOSA '19] within a time bound that is linear in the size of the cost matrix and polynomial in $C/δ$; here $C$ is the largest value in the cost matrix and $δ$ is the additive error. We present an adaptation of the classical graph algorithm of Gabow and Tarjan and provide a novel analysis of this algorithm that bounds its execution time by $O(\frac{n^2 C}δ+ \frac{nC^2}{δ^2})$. Our algorithm is extremely simple and executes, for an arbitrarily small constant $\varepsilon$, only $\lfloor \frac{2C}{(1-\varepsilon)δ} floor + 1$ iterations, where each iteration consists only of a Dijkstra-type search followed by a depth-first search. We also provide empirical results that suggest our algorithm is competitive with respect to a sequential implementation of the Sinkhorn algorithm in execution time. Moreover, our algorithm quickly computes a solution for very small values of $δ$ whereas Sinkhorn algorithm slows down due to numerical instability.

Motivation & Objective

• To design a scalable, combinatorial algorithm for additive approximation of optimal transport with provable runtime bounds.
• To overcome numerical instability issues in existing iterative algorithms like Sinkhorn at small additive error δ.
• To provide a simple, iterative method based on Gabow and Tarjan’s cost-scaling algorithm with a refined analysis of execution time.
• To enable efficient computation of optimal transport for geometric and metric costs using dynamic nearest neighbor data structures.
• To explore the feasibility of parallel and approximate nearest neighbor-based acceleration for large-scale optimal transport.

Proposed method

• Adapts Gabow and Tarjan’s cost-scaling algorithm for the transportation problem using integer scaling of demands, supplies, and costs.
• Executes a fixed number of iterations—⌊2C/((1−ε)δ)⌋+1—each consisting of a Dijkstra-type search followed by a depth-first search on the residual graph.
• Uses a scaled cost function c̄(·,·) obtained by dividing original costs by δ′ and rounding down, preserving correctness via a modified critical lemma.
• Employs a dynamic weighted nearest neighbor data structure to accelerate Dijkstra and DFS steps in geometric settings with poly-logarithmic query/update time.
• Analyzes runtime using a diameter-sensitive approach, bounding execution time by O(n²C/δ + nC²/δ²).
• Implements a sequential variant that avoids iterative row/column updates, differing from Sinkhorn and Greenkhorn.

Experimental results

Research questions

• RQ1 Can a combinatorial algorithm achieve δ-close additive approximation of optimal transport in time sublinear in C/δ?
• RQ2 Can Gabow and Tarjan’s cost-scaling framework be adapted to provide provable additive error bounds with minimal iteration count?
• RQ3 How does the performance of this algorithm compare to iterative methods like Sinkhorn in terms of execution time and solution quality at small δ?
• RQ4 Can dynamic nearest neighbor data structures be leveraged to achieve near-optimal time complexity for geometric optimal transport?
• RQ5 Is it possible to design a parallelizable combinatorial algorithm for δ-close optimal transport with low overhead?

Key findings

• The proposed algorithm runs in O(n²C/δ + nC²/δ²) time, providing a δ-close additive approximation to the optimal transport cost.
• The algorithm executes only ⌊2C/((1−ε)δ)⌋+1 iterations, each involving one Dijkstra and one DFS, making it highly efficient and simple.
• Empirical results show the algorithm is faster than the Sinkhorn algorithm across all δ values tested, especially for small δ, where Sinkhorn suffers from numerical instability.
• For δ = 0.025 to 0.2, the algorithm consistently outperforms Sinkhorn in execution time and produces lower-cost solutions, even when Sinkhorn is given a 5× larger error tolerance.
• When combined with dynamic nearest neighbor data structures, the algorithm achieves Õ(n(C/δ)²) time complexity for geometric costs like Euclidean and squared-Euclidean distance.
• The algorithm remains stable and efficient even at very small δ values, unlike Sinkhorn, which slows down significantly due to numerical issues.