[Paper Review] Tensor optimal transport, distance between sets of measures and tensor scaling

Shmuel Friedland|arXiv (Cornell University)|May 2, 2020

Markov Chains and Monte Carlo Methods34 references4 citations

TL;DR

This paper introduces tensor optimal transport (TOT) for $d > 2$ discrete probability measures, generalizing the matrix case via entropic regularization and a novel Sinkhorn-type algorithm. It proves geometric convergence of the algorithm to an $
$-approximate solution in $O(\omega^3 d^4 n^{d+1} \log n / \delta^3)$ operations, extending classical optimal transport to higher-order structures with provable complexity and convergence guarantees.

ABSTRACT

We study the optimal transport problem for $d>2$ discrete measures. This is a linear programming problem on $d$-tensors. It gives a way to compute a "distance" between two sets of discrete measures. We introduce an entropic regularization term, which gives rise to a scaling of tensors. We give a variation of the celebrated Sinkhorn scaling algorithm. We show that this algorithm can be viewed as a partial minimization algorithm of a strictly convex function. Under appropriate conditions the rate of convergence is geometric and we estimate the rate. Our results are generalizations of known results for the classical case of two discrete measures.

Motivation & Objective

• To generalize optimal transport from two to $d > 2$ discrete probability measures using $d$-tensors.
• To define a distance between two sets of $d$ discrete probability measures via a linear programming formulation on nonnegative tensors.
• To develop a computationally efficient approximation method using entropic regularization and tensor scaling.
• To establish geometric convergence rates for the proposed algorithm under strict convexity and partial minimization framework.
• To extend known results from matrix optimal transport (e.g., Sinkhorn algorithm) to the tensor setting with rigorous complexity and approximation error bounds.

Proposed method

• Formulates the tensor optimal transport (TOT) problem as a linear program over nonnegative $d$-tensors with marginal constraints on each mode.
• Introduces entropic regularization to the TOT problem, transforming it into a strictly convex optimization problem amenable to iterative scaling.
• Proposes a variation of the Sinkhorn algorithm for tensors, operating via alternating minimization over marginal constraints.
• Uses partial minimization of a strictly convex function to analyze convergence, linking the algorithm to iterative scaling of nonnegative tensors.
• Employs tensor contraction and scaling operations via $
  $-mode unfolding and marginalization to enforce consistency across all $d$ modes.
• Derives convergence rates by analyzing the Hessian and gradient behavior of the regularized objective, showing geometric convergence under appropriate conditions.

Experimental results

Research questions

• RQ1 How can optimal transport be generalized from two to $d > 2$ discrete probability measures using higher-order tensors?
• RQ2 What is the computational complexity of approximating the tensor optimal transport problem with a given accuracy $\delta$?
• RQ3 Can the Sinkhorn algorithm be extended to $d$-tensors with provable convergence and rate guarantees?
• RQ4 How does entropic regularization affect the structure and convergence of the tensor scaling process?
• RQ5 What is the relationship between the partial minimization of a strictly convex function and the iterative scaling algorithm in the tensor setting?

Key findings

• The proposed algorithm computes a $\delta$-approximate solution to the tensor optimal transport problem in $O(\omega^3 d^4 n^{d+1} \log n / \delta^3)$ operations, where $\omega$ is the range of the cost tensor entries.
• The algorithm converges geometrically to the unique minimizer of the regularized objective function, with the rate bounded by the strong convexity of the potential function.
• The tensor scaling process is equivalent to a partial minimization algorithm applied to a strictly convex function, ensuring global convergence.
• The iterates of the algorithm correspond to successive projections onto affine subspaces defined by marginal constraints, with convergence to a tensor in $\mathrm{U}(P)$, the set of feasible transport plans.
• The final output $\mathcal{A}_\star$ is a probability tensor satisfying all marginal constraints $\mathcal{A}_\star \times_k \mathcal{J}_{d-1} = \mathbf{p}_k$ for all $k \in [d]$, confirming feasibility.
• The convergence of the algorithm is shown to be geometric, with the rate depending on the condition number of the Hessian of the regularized objective.