[Paper Review] How to Train Your Energy-Based Models

Yang Song, Diederik P. Kingma|arXiv (Cornell University)|Jan 9, 2021

Energy Load and Power ForecastingEngineering82 references81 citations

TL;DR

The paper provides a friendly tutorial on training energy-based models (EBMs) using three main approaches: maximum likelihood with MCMC, Score Matching (including DSM and SSM variants), and Noise Contrastive Estimation, plus connections among them.

ABSTRACT

Energy-Based Models (EBMs), also known as non-normalized probabilistic models, specify probability density or mass functions up to an unknown normalizing constant. Unlike most other probabilistic models, EBMs do not place a restriction on the tractability of the normalizing constant, thus are more flexible to parameterize and can model a more expressive family of probability distributions. However, the unknown normalizing constant of EBMs makes training particularly difficult. Our goal is to provide a friendly introduction to modern approaches for EBM training. We start by explaining maximum likelihood training with Markov chain Monte Carlo (MCMC), and proceed to elaborate on MCMC-free approaches, including Score Matching (SM) and Noise Constrastive Estimation (NCE). We highlight theoretical connections among these three approaches, and end with a brief survey on alternative training methods, which are still under active research. Our tutorial is targeted at an audience with basic understanding of generative models who want to apply EBMs or start a research project in this direction.

Motivation & Objective

• Explain why EBMs are flexible yet hard to train due to the unknown normalizing constant Z_theta.
• Present three main training paradigms for EBMs: maximum likelihood with MCMC, Score Matching, and Noise Contrastive Estimation.
• Explain theoretical connections among these approaches and outline other active research directions.

Proposed method

• Derive the MLE gradient for EBMs and show how to obtain an unbiased Monte Carlo estimate via sampling from p_theta(x).
• Introduce Langevin MCMC and its variants for sampling, including practical techniques like CD and persistent CD.
• Present Score Matching and its Fisher divergence formulation, including DSM and SSM variants to handle non-ideal data supports.
• Show how denoising and sliced score matching circumvent intractable second derivatives and improve scalability.
• Explain Noise Contrastive Estimation by contrasting the EBM with a known-noise distribution and explain Z_theta as a learnable scalar.

Experimental results

Research questions

• RQ1 How can EBMs be trained effectively when the normalizing constant Z_theta is intractable?
• RQ2 How do MLE with MCMC, Score Matching, and Noise Contrastive Estimation relate to and complement each other in EBM training?
• RQ3 What practical techniques (e.g., CD, Langevin dynamics, DSM, SSM, NCE) enable scalable EBM learning?
• RQ4 How do score-based generative modeling approaches leverage score networks and multi-scale noise for high-quality sampling?

Key findings

• MLE with MCMC enables likelihood-based training by unbiased gradient estimation using samples from p_theta(x).
• Langevin MCMC and related methods provide practical sampling for EBMs and connect to CD with potential bias considerations.
• Score Matching and its variants (DSM, SSM) offer alternative training objectives that avoid explicit Z_theta, with trade-offs in consistency and computational cost.
• SSM provides a consistent alternative to DSM with linear-cost second-derivative terms, enabling scalable training for high-dimensional data.
• Score-based generative modeling uses noise-conditioning to jointly model scores across scales, improving sample quality for images and audio.
• NCE furnishes another pathway by contrastive learning against a known noise distribution, with Z_theta treated as a learnable scalar.