[Paper Review] Fixed-Support Wasserstein Barycenters: Computational Hardness and Fast Algorithm

Tianyi Lin, Nhat Ho+3 more|arXiv (Cornell University)|Feb 12, 2020

Geometric Analysis and Curvature Flows62 references18 citations

TL;DR

This paper establishes that the fixed-support Wasserstein barycenter problem (FS-WBP) is computationally hard to solve via minimum-cost flow methods when $ m \geq 3 $ and $ n \geq 3 $, as the constraint matrix is not totally unimodular. It proposes FastIBP, a provably faster deterministic variant of the iterative Bregman projection algorithm with a complexity bound of $ \widetilde{O}(mn^{7/3}\varepsilon^{-4/3}) $, outperforming existing methods in both $ \varepsilon $-dependence and $ n $-dependence.

ABSTRACT

We study the fixed-support Wasserstein barycenter problem (FS-WBP), which consists in computing the Wasserstein barycenter of $m$ discrete probability measures supported on a finite metric space of size $n$. We show first that the constraint matrix arising from the standard linear programming (LP) representation of the FS-WBP is extit{not totally unimodular} when $m \geq 3$ and $n \geq 3$. This result resolves an open question pertaining to the relationship between the FS-WBP and the minimum-cost flow (MCF) problem since it proves that the FS-WBP in the standard LP form is not an MCF problem when $m \geq 3$ and $n \geq 3$. We also develop a provably fast extit{deterministic} variant of the celebrated iterative Bregman projection (IBP) algorithm, named extsc{FastIBP}, with a complexity bound of $ ilde{O}(mn^{7/3}\varepsilon^{-4/3})$, where $\varepsilon \in (0, 1)$ is the desired tolerance. This complexity bound is better than the best known complexity bound of $ ilde{O}(mn^2\varepsilon^{-2})$ for the IBP algorithm in terms of $\varepsilon$, and that of $ ilde{O}(mn^{5/2}\varepsilon^{-1})$ from accelerated alternating minimization algorithm or accelerated primal-dual adaptive gradient algorithm in terms of $n$. Finally, we conduct extensive experiments with both synthetic data and real images and demonstrate the favorable performance of the extsc{FastIBP} algorithm in practice.

Motivation & Objective

• To resolve the open question on whether the fixed-support Wasserstein barycenter problem (FS-WBP) can be reduced to a minimum-cost flow (MCF) problem in standard linear programming form.
• To develop a provably faster and deterministic algorithm for solving FS-WBP, especially for large-scale problems with $ m \geq 3 $ and $ n \geq 3 $.
• To improve the theoretical complexity bounds of existing iterative Bregman projection (IBP) and accelerated algorithms in terms of $ \varepsilon $ and $ n $.
• To empirically validate the performance of the proposed FastIBP algorithm on synthetic data and real-world image datasets like MNIST.

Proposed method

• Prove that the constraint matrix of the standard LP formulation of FS-WBP is not totally unimodular when $ m \geq 3 $ and $ n \geq 3 $, thus ruling out MCF reformulation.
• Propose FastIBP, a deterministic variant of the iterative Bregman projection (IBP) algorithm, using a novel step-size selection strategy to improve convergence.
• Establish a theoretical complexity bound of $ \widetilde{O}(mn^{7/3}\varepsilon^{-4/3}) $ for FastIBP, improving upon the $ \widetilde{O}(mn^{2}\varepsilon^{-2}) $ bound of standard IBP in $ \varepsilon $.
• Use a primal-dual framework to derive complexity bounds, leveraging recent advances in accelerated gradient methods for optimal transport.
• Implement and evaluate FastIBP on synthetic data and MNIST images, comparing objective value, feasibility, and runtime against Gurobi and standard IBP.
• Conduct ablation studies with varying $ m $ and $ \varepsilon $, demonstrating linear scalability of FastIBP in $ m $ and superior convergence in $ \varepsilon $.

Experimental results

Research questions

• RQ1 Is the fixed-support Wasserstein barycenter problem (FS-WBP) equivalent to a minimum-cost flow (MCF) problem in the standard LP formulation when $ m \geq 3 $ and $ n \geq 3 $?
• RQ2 Can a deterministic variant of the iterative Bregman projection (IBP) algorithm achieve better complexity bounds than existing accelerated methods in terms of $ \varepsilon $ and $ n $?
• RQ3 Does the proposed FastIBP algorithm outperform standard IBP and commercial solvers like Gurobi in terms of runtime and solution quality on large-scale and real-world datasets?
• RQ4 Why does the dual simplex method perform poorly on FS-WBP despite its theoretical guarantees, and how does this relate to the problem's structural hardness?

Key findings

• The constraint matrix of the standard LP formulation of FS-WBP is not totally unimodular when $ m \geq 3 $ and $ n \geq 3 $, proving that FS-WBP is not a minimum-cost flow problem in this regime.
• FastIBP achieves a complexity bound of $ \widetilde{O}(mn^{7/3}\varepsilon^{-4/3}) $, which improves upon the $ \widetilde{O}(mn^{2}\varepsilon^{-2}) $ bound of standard IBP in terms of $ \varepsilon $.
• FastIBP outperforms the $ \widetilde{O}(mn^{5/2}\varepsilon^{-1}) $ complexity of accelerated primal-dual and alternating minimization methods in terms of $ n $-dependence.
• On synthetic data with $ n=100 $, FastIBP scales linearly with $ m $, while Gurobi’s runtime grows much more rapidly, confirming the algorithmic advantage of FastIBP.
• On MNIST, FastIBP produces sharper barycenters than standard IBP at $ \eta = 0.001 $, indicating higher-quality solutions with better visual fidelity.
• The feasibility violation of Gurobi is worse than FastIBP’s in the same experiments, and Gurobi’s runtime grows faster than linear, suggesting structural inefficiency of dual simplex on FS-WBP.