[Paper Review] Deep Generative Learning via Schrödinger Bridge

Gefei Wang, Yuling Jiao+3 more|arXiv (Cornell University)|Jun 19, 2021

Generative Adversarial Networks and Image Synthesis38 references4 citations

TL;DR

This paper proposes a novel deep generative modeling framework based on the Schrödinger Bridge, which formulates generative learning as entropy interpolation between a reference and target distribution via a time-inhomogeneous SDE. Under mild smoothness assumptions, the method ensures theoretical consistency and achieves performance competitive with state-of-the-art GANs on synthetic and real-world benchmarks, demonstrating strong results in image generation, interpolation, and inpainting.

ABSTRACT

We propose to learn a generative model via entropy interpolation with a Schrödinger Bridge. The generative learning task can be formulated as interpolating between a reference distribution and a target distribution based on the Kullback-Leibler divergence. At the population level, this entropy interpolation is characterized via an SDE on $[0,1]$ with a time-varying drift term. At the sample level, we derive our Schrödinger Bridge algorithm by plugging the drift term estimated by a deep score estimator and a deep density ratio estimator into the Euler-Maruyama method. Under some mild smoothness assumptions of the target distribution, we prove the consistency of both the score estimator and the density ratio estimator, and then establish the consistency of the proposed Schrödinger Bridge approach. Our theoretical results guarantee that the distribution learned by our approach converges to the target distribution. Experimental results on multimodal synthetic data and benchmark data support our theoretical findings and indicate that the generative model via Schrödinger Bridge is comparable with state-of-the-art GANs, suggesting a new formulation of generative learning. We demonstrate its usefulness in image interpolation and image inpainting.

Motivation & Objective

• To address the gap between empirical success and theoretical consistency in deep generative models, particularly for likelihood-based and implicit models.
• To develop a theoretically grounded generative modeling approach that avoids strong assumptions like log-concavity or log-Sobolev inequality.
• To enable stable learning of multimodal distributions, overcoming common GAN failures such as mode collapse.
• To propose a two-stage algorithm that first learns a smoothed target distribution and then refines it to match the true data distribution.
• To demonstrate practical utility through image interpolation and inpainting tasks, showing competitiveness with state-of-the-art GANs.

Proposed method

• Formulates generative learning as entropy interpolation between reference and target distributions using the Kullback-Leibler divergence.
• Characterizes the population-level solution via a stochastic differential equation (SDE) on [0,1] with a time-varying drift term.
• At the sample level, implements the Euler-Maruyama method using a deep score estimator and a deep density ratio estimator to approximate the drift.
• Employs a two-stage training procedure: first learning a smoothed version of the target distribution, then refining it to match the true distribution.
• Uses a score network to estimate the score function and a residual network to estimate the density ratio, with proper normalization and data preprocessing.
• Applies a final denoising step after generation to reduce noise in the output samples.

Experimental results

Research questions

• RQ1 Can a Schrödinger Bridge formulation achieve theoretical consistency in deep generative modeling without requiring strong assumptions like log-concavity?
• RQ2 How does the proposed two-stage algorithm improve stability and performance in learning multimodal data distributions?
• RQ3 Can the Schrödinger Bridge approach match or exceed the performance of state-of-the-art GANs on benchmark datasets like CIFAR-10 and CelebA?
• RQ4 What is the impact of different noise levels and time parameters on the quality of generated samples?
• RQ5 How effective is the method in downstream tasks such as image interpolation and image inpainting?

Key findings

• The proposed Schrödinger Bridge approach achieves theoretical consistency under mild smoothness assumptions, guaranteeing convergence to the target distribution.
• On CIFAR-10 with σ²=2.0, the method achieves a Fréchet Inception Distance (FID) of 12.45 and an Inception Score of 7.98 with τ=3.5.
• For σ²=5.0, the FID reaches 15.41 and Inception Score 7.59 at τ=8.0, indicating robust performance across noise scales.
• The method successfully learns multimodal synthetic distributions without mode collapse, in contrast to GANs which often fail in such settings.
• Image interpolation and inpainting results demonstrate the method's practical utility and generalization capability.
• The final denoising step significantly improves sample quality by reducing residual noise in generated images.