[Paper Review] Classifier-Free Diffusion Guidance

Jonathan Ho, Tim Salimans|arXiv (Cornell University)|Jul 26, 2022

Model Reduction and Neural NetworksPhysics and Astronomy731 citations

TL;DR

The paper introduces classifier-free guidance, a method to trade off sample fidelity and diversity in diffusion models by jointly training conditional and unconditional models and combining their score estimates, eliminating the need for an auxiliary classifier.

ABSTRACT

Classifier guidance is a recently introduced method to trade off mode coverage and sample fidelity in conditional diffusion models post training, in the same spirit as low temperature sampling or truncation in other types of generative models. Classifier guidance combines the score estimate of a diffusion model with the gradient of an image classifier and thereby requires training an image classifier separate from the diffusion model. It also raises the question of whether guidance can be performed without a classifier. We show that guidance can be indeed performed by a pure generative model without such a classifier: in what we call classifier-free guidance, we jointly train a conditional and an unconditional diffusion model, and we combine the resulting conditional and unconditional score estimates to attain a trade-off between sample quality and diversity similar to that obtained using classifier guidance.

Motivation & Objective

• Motivate reducing reliance on external classifiers for guidance in diffusion models.
• Demonstrate that guidance can be achieved with a pure generative model by jointly training conditional and unconditional diffusion models.
• Show that varying guidance strength yields a trade-off between sample quality (IS) and diversity (FID).
• Provide a simple, practical training and sampling procedure for classifier-free guidance that rivals classifier-guided methods.

Proposed method

• Train a diffusion model with both conditional and unconditional branches in a single network by randomly dropping conditioning with probability p_uncond.
• Predict denoising scores ε_θ(z_λ, c) and ε_θ(z_λ) and form a guided score ε̃_t = (1+w)ε_θ(z_λ, c) − wε_θ(z_λ) during sampling (classifier-free guidance).
• Use a joint objective to train both conditional and unconditional components via denoising score matching over multiple noise scales λ.
• During training, sample λ from a cosine-based schedule and optimize ε_θ to match the true ε for corrupted zλ.
• At inference, sample with a fixed mix weight w to trade off fidelity vs. diversity.
• Discuss that the approach relies on unconstrained networks, so the guided score may not correspond to a gradient of any explicit classifier.

Experimental results

Research questions

• RQ1 Can guidance be achieved in diffusion models without training a separate classifier?
• RQ2 Does mixing unconditional and conditional score Estimates provide a controllable IS/FID trade-off similar to classifier guidance?
• RQ3 How does classifier-free guidance compare in simplicity, training requirements, and sampling efficiency to classifier-guided diffusion?
• RQ4 What are the effects of training unconditional probability p_uncond and guidance strength w on image quality and diversity?

Key findings

• Classifier-free guidance achieves a controllable IS/FID trade-off similar to classifier guidance by adjusting the guidance strength w.
• For 64×64 ImageNet, small w yields best FID, while larger w improves Inception Score, illustrating a clear fidelity-diversity trade-off.
• For 128×128 ImageNet, the method attains competitive FID and IS across varying timesteps T; at w values like 0.3–0.4 it achieves strong IS with good FID.
• Training with p_uncond small (e.g., 0.1–0.2) suffices, showing only a modest unconditional capacity is needed for effective guidance.
• The strongest guided samples show higher fidelity but reduced diversity, aligning with the intended trade-off between quality and variety.
• The approach matches or surpasses existing methods on reported benchmarks, with competitive or superior IS/FID at comparable sampling budgets.