[Paper Review] Large-Scale Optimal Transport and Mapping Estimation

Vivien Seguy, Bharath Bhushan Damodaran+4 more|arXiv (Cornell University)|Nov 7, 2017

Traffic Prediction and Management Techniques35 references79 citations

TL;DR

The paper introduces a two-step framework to learn an optimal map between distributions: (1) compute a large-scale regularized OT plan via a stochastic dual method, and (2) extract a Monge map as a neural network from the barycentric projection, with theoretical convergence guarantees and applications to domain adaptation and generative modeling.

ABSTRACT

This paper presents a novel two-step approach for the fundamental problem of learning an optimal map from one distribution to another. First, we learn an optimal transport (OT) plan, which can be thought as a one-to-many map between the two distributions. To that end, we propose a stochastic dual approach of regularized OT, and show empirically that it scales better than a recent related approach when the amount of samples is very large. Second, we estimate a extit{Monge map} as a deep neural network learned by approximating the barycentric projection of the previously-obtained OT plan. This parameterization allows generalization of the mapping outside the support of the input measure. We prove two theoretical stability results of regularized OT which show that our estimations converge to the OT plan and Monge map between the underlying continuous measures. We showcase our proposed approach on two applications: domain adaptation and generative modeling.

Motivation & Objective

• Motivate the problem of mapping one distribution to another for applications like domain adaptation and generative modeling.
• Develop a scalable two-step method to first learn an OT plan and then recover an optimal Monge map.
• Provide theoretical results on the convergence of regularized OT plans and their barycentric projections to the true Monge map between continuous measures.
• Demonstrate the method on domain adaptation and large-scale generative modeling tasks.

Proposed method

• Use entropy or L2 regularization to relax OT and enable a dual formulation with a concave objective.
• Optimize the regularized OT dual via stochastic gradient ascent, enabling scalability to large sample sizes.
• Recover a Monge map by computing the barycentric projection of the OT plan and parameterize it with a neural network to generalize beyond the input support.
• Learn the mapping by minimizing a weighted squared-error objective between Y and f_theta(X) using H_e weights arising from the regularized plan.
• Prove convergence results: entropy-regularized plans converge to the true OT plan; barycentric projections converge to the Monge map for underlying continuous measures (Theorems 1 and 2).

Experimental results

Research questions

• RQ1 How can optimal transport be computed efficiently for very large-scale discrete or continuous measures?
• RQ2 Can the barycentric projection of a regularized OT plan be used to recover a Monge map that generalizes outside the input support?
• RQ3 Do the regularized OT plans and their barycentric projections converge to the true OT plan and Monge map as sample size grows and regularization vanishes?
• RQ4 How effective is the proposed two-step OT mapping framework for practical tasks such as domain adaptation and generative modeling?

Key findings

• The dual stochastic OT algorithm converges faster than a stochastic semi-dual approach on large-scale OT problems.
• The method yields improved domain adaptation performance over prior OT-based approaches on MNIST/USPS/SVHN benchmarks, with Monge-map-based approaches achieving higher accuracies.
• Among regularizations, entropy and L2 yield competitive results, with Monge map learning (Alg. 1+2) achieving the best performance on MNIST→USPS (77.92%) in the reported setup.
• The barycentric projection of the regularized OT plan converges to the true Monge map between the underlying continuous measures as sample size grows and regularization vanishes (Theorems 1–2).
• Corollaries show the mapped source measure converges to the target measure, supporting use in domain adaptation and generative modeling.