[Paper Review] Learning Non-Convergent Non-Persistent Short-Run MCMC Toward Energy-Based Model

Erik Nijkamp, Mitch Hill+2 more|arXiv (Cornell University)|Apr 22, 2019

Generative Adversarial Networks and Image Synthesis59 references72 citations

TL;DR

The paper studies learning EBMs with non-convergent short-run MCMC, showing the resulting learned short-run MCMC can generate realistic images and function as a generator or flow model, even though it learns a wrong sampler of a wrong model.

ABSTRACT

This paper studies a curious phenomenon in learning energy-based model (EBM) using MCMC. In each learning iteration, we generate synthesized examples by running a non-convergent, non-mixing, and non-persistent short-run MCMC toward the current model, always starting from the same initial distribution such as uniform noise distribution, and always running a fixed number of MCMC steps. After generating synthesized examples, we then update the model parameters according to the maximum likelihood learning gradient, as if the synthesized examples are fair samples from the current model. We treat this non-convergent short-run MCMC as a learned generator model or a flow model. We provide arguments for treating the learned non-convergent short-run MCMC as a valid model. We show that the learned short-run MCMC is capable of generating realistic images. More interestingly, unlike traditional EBM or MCMC, the learned short-run MCMC is capable of reconstructing observed images and interpolating between images, like generator or flow models. The code can be found in the Appendix.

Motivation & Objective

• Motivate and analyze learning energy-based models (EBMs) using non-convergent, non-mixing short-run MCMC.
• Demonstrate that a fixed-step, noise-initialized MCMC can yield realistic image generation.
• Show that the learned short-run MCMC can interpolate between images and reconstruct observed images.
• Explain theoretical connections to generalized moment matching and entropy considerations.
• Propose that short-run MCMC can be viewed as a valid generator model and inform related tasks (inpainting, super-resolution, style transfer).

Proposed method

• Define p_theta(x) as a Gibbs distribution with energy f_theta(x) parameterized by a ConvNet.
• Replace exact sampling from p_theta with a fixed-K step MCMC M_theta starting from a fixed p0 (e.g., uniform noise) to induce q_theta.
• Update theta via the maximum likelihood gradient using data expectations minus q_theta expectations (Δ(theta)).
• Inject Gaussian noise into observed data to stabilize learning and promote convergence of the estimating equation.
• Interpret q_theta as a generator or flow model with x = M_theta(z, u) where z ~ p0 and u is MCMC randomness.
• Provide a conceptual and mathematical link to generalized moment matching estimators and information-theoretic dualities (Pythagorean relations).

Experimental results

Research questions

• RQ1 Can non-convergent, non-persistent short-run MCMC effectively learn an energy-based model?
• RQ2 Is the resulting short-run MCMC a valid model for data and capable of generation, interpolation, and reconstruction?
• RQ3 How do hyperparameters (K, noise level, model width) affect synthesis quality and stability?
• RQ4 Can the learned short-run MCMC be interpreted as a generator or flow model with latent variables and a residual architecture?
• RQ5 What are the theoretical connections to moment matching and entropy within this learning scheme?

Key findings

• Short-run MCMC with fixed initialization can generate realistic images despite non-convergence.
• The learned short-run MCMC can interpolate between generated samples and reconstruct observed images, akin to generator/flow models.
• Increasing K improves fidelity and reduces KL divergence between q_theta and p_theta, up to computational limits.
• Adding noise to data stabilizes training and allows convergence of the estimating equation Δ(theta)=0.
• Synthesis quality improves with more features (n_f) and appropriate noise levels; IS/FID metrics show competitive results.