[Paper Review] Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm

Qiang Liu, Dilin Wang|arXiv (Cornell University)|Aug 16, 2016

Gaussian Processes and Bayesian Inference33 references325 citations

TL;DR

SVGD transports a set of particles to the target distribution by a functional gradient descent in RKHS, providing a general-purpose variational inference method that reduces to MAP with one particle.

ABSTRACT

We propose a general purpose variational inference algorithm that forms a natural counterpart of gradient descent for optimization. Our method iteratively transports a set of particles to match the target distribution, by applying a form of functional gradient descent that minimizes the KL divergence. Empirical studies are performed on various real world models and datasets, on which our method is competitive with existing state-of-the-art methods. The derivation of our method is based on a new theoretical result that connects the derivative of KL divergence under smooth transforms with Stein's identity and a recently proposed kernelized Stein discrepancy, which is of independent interest.

Motivation & Objective

• Motivate the need for a general, easy-to-use variational inference tool that can handle diverse models and large datasets.
• Develop a particle-based variational framework that optimizes KL divergence without requiring explicit normalization constants.
• Derive a closed-form optimal perturbation direction using kernelized Stein discrepancy and RKHS.
• Provide an implementable algorithm that mirrors gradient descent and scales beyond model-specific VI methods.

Proposed method

• Represent the variational family as transformed samples z=T(x) from a reference distribution.
• Use small perturbations T(x)=x+εφ(x) and apply Stein’s identity to relate KL derivative to the Stein operator.
• Define the kernelized Stein discrepancy (KSD) to obtain a closed-form optimal perturbation φ* in the RKHS.
• Derive that the steepest descent direction φ* is φ*_{q,p}(·)=E_{x~q}[k(x,·)∇_x log p(x)+∇_x k(x,·)].
• Provide the practical SVGD update x_i^{ℓ+1} = x_i^ℓ + ε_ℓ φ̂^*(x_i^ℓ) with φ̂^*(x) estimated from particles.
• Discuss computational considerations including mini-batching for ∇_x log p and kernel matrix handling.

Experimental results

Research questions

• RQ1 How can KL divergence be minimized via smooth transforms without computing the normalization constant Z?
• RQ2 What is the optimal perturbation direction in RKHS that yields the steepest descent of KL divergence for a given current distribution?
• RQ3 Can a simple, general-purpose particle-based variational method achieve competitive performance across diverse Bayesian models?
• RQ4 How does SVGD relate to MAP when using a single particle and how does it scale with multiple particles?
• RQ5 What are the practical considerations (bandwidth, batching, efficiency) for implementing SVGD on large datasets?

Key findings

• The proposed algorithm yields competitive results versus state-of-the-art methods across toy and real-world models.
• With a single particle, SVGD reduces to gradient ascent for MAP, illustrating its generalization beyond traditional VI.
• The particle update combines a smoothed gradient toward high-probability regions with a repulsive term to maintain diversity.
• Kernelized Stein discrepancy provides a principled, closed-form direction for the steepest KL descent within the RKHS unit ball.
• Efficiency is achieved through minibatch gradient estimates and parallelizable particle updates, making SVGD scalable to large datasets.
• Code for SVGD is publicly available at the authors’ repository.