[Paper Review] Deep Generative Learning via Schr\"{o}dinger Bridge

Gefei Wang, Yuling Jiao+3 more|arXiv (Cornell University)|Jun 19, 2021

Generative Adversarial Networks and Image Synthesis58 references4 citations

TL;DR

This paper proposes a deep generative modeling approach using the Schrödinger Bridge framework to interpolate between a reference and target distribution via entropy minimization. By estimating time-varying drift with deep score and density ratio models, and applying the Euler–Maruyama scheme, the method achieves consistent convergence to the target distribution, with empirical results showing performance competitive with state-of-the-art GANs on image generation, interpolation, and inpainting tasks.

ABSTRACT

We propose to learn a generative model via entropy interpolation with a Schrodinger Bridge. The generative learning task can be formulated as interpolating between a reference distribution and a target distribution based on the KullbackLeibler divergence. At the population level, this entropy interpolation is characterized via an SDE on [0, 1] with a time-varying drift term. At the sample level, we derive our Schrodinger Bridge algorithm by plugging the drift term estimated by a deep score estimator and a deep density ratio estimator into the Euler-Maruyama method. Under some mild smoothness assumptions of the target distribution, we prove the consistency of both the score estimator and the density ratio estimator, and then establish the consistency of the proposed Schrodinger Bridge approach. Our theoretical results guarantee that the distribution learned by our approach converges to the target distribution. Experimental results on multimodal synthetic data and benchmark data support our theoretical findings and indicate that the generative model via Schrodinger Bridge is comparable with state-of-the-art GANs, suggesting a new formulation of generative learning. We demonstrate its usefulness in image interpolation and image inpainting.

Motivation & Objective

• To develop a theoretically grounded deep generative model based on entropy interpolation between reference and target distributions.
• To address the challenge of learning complex target distributions by formulating the problem as a Schrödinger Bridge process with time-varying drift.
• To ensure consistency of the learned generative model under mild smoothness assumptions of the target distribution.
• To demonstrate the method’s effectiveness in image interpolation and image inpainting tasks, showing practical utility beyond synthetic data.

Proposed method

• Formulate the generative learning task as minimizing the Kullback–Leibler divergence between the interpolated process and the target distribution at time T.
• Model the interpolation process as a stochastic differential equation (SDE) on [0,1] with a time-varying drift term derived from the Schrödinger Bridge framework.
• Estimate the drift term using a deep score estimator trained on the target data distribution.
• Simultaneously estimate the density ratio between the target and reference distributions using a deep neural network.
• Implement the Euler–Maruyama discretization of the SDE using the estimated drift and diffusion terms to generate samples.
• Train the score and density ratio estimators end-to-end using gradient-based optimization to minimize the KL divergence objective.

Experimental results

Research questions

• RQ1 Can the Schrödinger Bridge framework be effectively adapted to deep generative modeling using neural network-based estimators?
• RQ2 Does the proposed method achieve consistent convergence to the target distribution under mild smoothness assumptions?
• RQ3 How does the performance of the Schrödinger Bridge generative model compare to state-of-the-art GANs on standard benchmarks?
• RQ4 Can the method be successfully applied to downstream tasks such as image interpolation and image inpainting?

Key findings

• The proposed Schrödinger Bridge method achieves consistent convergence to the target distribution under mild smoothness assumptions on the target density.
• The deep score estimator and density ratio estimator are proven to be consistent, ensuring the overall method’s statistical reliability.
• Empirical results on multimodal synthetic data confirm the theoretical convergence properties of the framework.
• On benchmark datasets, the method achieves generative performance comparable to state-of-the-art GANs in both sample quality and diversity.
• The method demonstrates strong performance in image interpolation and image inpainting, highlighting its practical utility beyond standard generation tasks.