[Paper Review] Flow Matching for Generative Modeling

Yaron Lipman, Ricky T. Q. Chen+3 more|arXiv (Cornell University)|Oct 6, 2022

Generative Adversarial Networks and Image Synthesis86 citations

TL;DR

A simulation-free training framework for Continuous Normalizing Flows (CNFs) called Flow Matching (FM) that uses per-sample conditional probability paths, including an Optimal Transport (OT) path, to achieve scalable, efficient generation and improved likelihoods and sample quality versus diffusion-based methods.

ABSTRACT

We introduce a new paradigm for generative modeling built on Continuous Normalizing Flows (CNFs), allowing us to train CNFs at unprecedented scale. Specifically, we present the notion of Flow Matching (FM), a simulation-free approach for training CNFs based on regressing vector fields of fixed conditional probability paths. Flow Matching is compatible with a general family of Gaussian probability paths for transforming between noise and data samples -- which subsumes existing diffusion paths as specific instances. Interestingly, we find that employing FM with diffusion paths results in a more robust and stable alternative for training diffusion models. Furthermore, Flow Matching opens the door to training CNFs with other, non-diffusion probability paths. An instance of particular interest is using Optimal Transport (OT) displacement interpolation to define the conditional probability paths. These paths are more efficient than diffusion paths, provide faster training and sampling, and result in better generalization. Training CNFs using Flow Matching on ImageNet leads to consistently better performance than alternative diffusion-based methods in terms of both likelihood and sample quality, and allows fast and reliable sample generation using off-the-shelf numerical ODE solvers.

Motivation & Objective

• Develop a scalable, simulation-free training objective for Continuous Normalizing Flows (CNFs).
• Leverage per-sample conditional probability paths to construct tractable targets for CNF training.
• Explore a general family of probability paths (including diffusion and OT) within Flow Matching.
• Demonstrate that Flow Matching can outperform diffusion-based methods on image datasets in likelihood and sample quality.
• Show that OT-based paths yield faster training, sampling, and better generalization.

Proposed method

• Define Flow Matching (FM) objective to regress a neural vector field v_t to a target path-generating field u_t.
• Construct p_t and u_t from conditional probability paths p_t(x|x1) and conditional vector fields u_t(x|x1) and aggregate to obtain marginal p_t and u_t.
• Use Conditional Flow Matching (CFM), which has equivalent gradients to FM, enabling per-sample training without explicit marginal targets.
• Adopt a general Gaussian conditional path p_t(x|x1) with mean mu_t(x1) and std sigma_t(x1), and derive the conditional vector field u_t(x|x1) via the flow map psi_t.
• Specialize to diffusion-based paths (VE and VP) and Optimal Transport (OT) displacement interpolants, highlighting OT’s linear, straight-line trajectories and simpler regression targets.
• Train CNFs on ImageNet with Flow Matching (FM) using OT paths and compare to diffusion-based baselines on likelihood (NLL/BPD) and FID, plus sampling efficiency.

Experimental results

Research questions

• RQ1 Can a simulation-free Flow Matching objective train CNFs at scale without solving ODEs for each step?
• RQ2 How do conditional probability paths (diffusion vs OT) compare in terms of training stability, sampling efficiency, and model quality?
• RQ3 Does the OT-based conditional path provide faster training and better generalization than diffusion paths when used in Flow Matching?
• RQ4 How does Flow Matching perform on large-scale datasets (ImageNet) in terms of likelihood and sample quality relative to diffusion-based methods?
• RQ5 Can Flow Matching enable reliable conditional generation and fast sampling with off-the-shelf ODE solvers?

Key findings

• Flow Matching with OT paths yields better NLL (BPD) and FID and often lower NFE than diffusion baselines across CIFAR-10 and ImageNet variants.
• FM-OT consistently achieves the best results among baselines on CIFAR-10 and ImageNet 32x32/64x64 in Table 1 (NLL, FID, NFE).
• On ImageNet-128x128, FM w/ OT achieves a competitive NLL (2.90) and FID (20.9) compared to a range of GAN-based methods listed, with Flow Matching offering strong likelihood and sample quality.
• Flow Matching with OT enables faster sampling: for the same numerical accuracy, OT paths require fewer function evaluations (NFE) than diffusion paths, and provide better cost-quality trade-offs.
• CFM provides equivalent gradients to FM, enabling tractable per-sample training without explicit marginal vector fields.
• OT paths produce straight-line trajectories in latent space, leading to simpler regression targets and more efficient training/sampling compared to diffusion paths.