[Paper Review] Sliced-Wasserstein Flows: Nonparametric Generative Modeling via Optimal Transport and Diffusions

Antoine Liutkus, Umut Ş Imşekli+3 more|arXiv (Cornell University)|Jun 21, 2018

Generative Adversarial Networks and Image SynthesisComputer Science42 citations

TL;DR

The paper introduces a parameter-free, nonparametric implicit generative modeling (IGM) algorithm based on a gradient flow in Wasserstein space, using sliced-Wasserstein distance and entropy regularization to learn distributions and sample from them with theoretical guarantees.

ABSTRACT

By building upon the recent theory that established the connection between implicit generative modeling (IGM) and optimal transport, in this study, we propose a novel parameter-free algorithm for learning the underlying distributions of complicated datasets and sampling from them. The proposed algorithm is based on a functional optimization problem, which aims at finding a measure that is close to the data distribution as much as possible and also expressive enough for generative modeling purposes. We formulate the problem as a gradient flow in the space of probability measures. The connections between gradient flows and stochastic differential equations let us develop a computationally efficient algorithm for solving the optimization problem. We provide formal theoretical analysis where we prove finite-time error guarantees for the proposed algorithm. To the best of our knowledge, the proposed algorithm is the first nonparametric IGM algorithm with explicit theoretical guarantees. Our experimental results support our theory and show that our algorithm is able to successfully capture the structure of different types of data distributions.

Motivation & Objective

• Motivate implicit generative modeling (IGM) and its OT connections.
• Develop a parameter-free, nonparametric learning algorithm with theoretical guarantees.
• Formulate a gradient flow in Wasserstein space to approximate the target distribution ν.
• Incorporate entropy regularization to ensure expressiveness and avoid overfitting to data.
• Provide a practical algorithm with finite-time error bounds and demonstrate on synthetic and real data.

Proposed method

• Formulate the learning problem as minimizing F^ν_λ(μ) = (1/2) SW_2^2(μ, ν) + λ H(μ).
• Use sliced-Wasserstein distance SW_2 which reduces high-dimensional OT to averages of 1D OT problems.
• Represent the evolution as a generalized minimizing movement in (P_2, W_2) with a PDE linked to a Fokker-Planck equation.
• Derive a stochastic particle system with drift v_t(x, μ_t) expressed via Kantorovich potentials between projected measures.
• Approximate the drift using Monte Carlo over random directions θ on the sphere, enabling an approximate Euler–Maruyama discretization.
• Show the connection to McKean–Vlasov type SDEs and provide finite-time error bounds for the discretized scheme.
• Implement Algorithm 1 (Sliced-Wasserstein Flow) to update particles with drift estimates and Gaussian noise.
• Provide theoretical results: existence of a gradient flow solution path and a finite-time bound on the total variation error between particle approximations and the target flow.

Experimental results

Research questions

• RQ1 Can a nonparametric, parameter-free IGM method be developed with explicit convergence guarantees?
• RQ2 Does the sliced-Wasserstein flow framework yield a well-defined gradient flow in Wasserstein space with entropy regularization?
• RQ3 Can a practical particle-based algorithm approximate the gradient flow efficiently, with finite-time error guarantees?
• RQ4 How does entropy regularization affect expressiveness and prevent overfitting to finite data?
• RQ5 Do experiments on synthetic and real data validate the theoretical guarantees and demonstrate learning/generation capabilities?

Key findings

• A gradient-flow-based, nonparametric IGM algorithm is proposed with finite-time error guarantees.
• The SW_2 distance plus entropy regularization yields a well-defined flow whose density evolves by a PDE linked to a Fokker–Planck equation.
• A practical particle system with an approximate Euler–Maruyama discretization is derived and shown to approximate the target flow under suitable conditions.
• The drift is estimated via Monte Carlo over random projection directions, enabling scalable computation.
• Algorithm experiments (Gaussian mixture, MNIST, CelebA bottleneck features) show decreasing SW cost and plausible sample generation, with regularization controlling spread.
• Theoretical results connect the method to SGLD-type dynamics and provide non-asymptotic error bounds in terms of step size, drift variance, and regularization parameter λ.