[Paper Review] A Non-Asymptotic Analysis for Stein Variational Gradient Descent

Anna Korba, Adil Salim+3 more|arXiv (Cornell University)|Jun 17, 2020

Markov Chains and Monte Carlo Methods35 references15 citations

TL;DR

This paper presents a non-asymptotic analysis of Stein Variational Gradient Descent (SVGD), establishing a descent lemma and convergence rates for the average Stein Fisher divergence (Kernel Stein Discrepancy) in the infinite-particle regime. It further provides a finite-particle propagation of chaos bound, quantifying the deviation of the empirical particle distribution from its population limit under kernel and potential regularity assumptions.

ABSTRACT

We study the Stein Variational Gradient Descent (SVGD) algorithm, which optimises a set of particles to approximate a target probability distribution $π\propto e^{-V}$ on $\mathbb{R}^d$. In the population limit, SVGD performs gradient descent in the space of probability distributions on the KL divergence with respect to $π$, where the gradient is smoothed through a kernel integral operator. In this paper, we provide a novel finite time analysis for the SVGD algorithm. We provide a descent lemma establishing that the algorithm decreases the objective at each iteration, and rates of convergence for the average Stein Fisher divergence (also referred to as Kernel Stein Discrepancy). We also provide a convergence result of the finite particle system corresponding to the practical implementation of SVGD to its population version.

Motivation & Objective

• To provide a finite-time, non-asymptotic analysis of SVGD, addressing the lack of quantitative convergence rates in existing literature.
• To establish a descent lemma showing that SVGD decreases the objective at each iteration with a constant step-size in the population limit.
• To derive convergence rates for the average Stein Fisher divergence (Kernel Stein Discrepancy) in the infinite-particle regime.
• To quantify the deviation of the finite-particle system from its population-level counterpart via a propagation of chaos bound.
• To lay theoretical groundwork for understanding the convergence behavior of practical SVGD implementations with finite particles.

Proposed method

• Formulates SVGD as gradient descent in the Wasserstein space of probability measures, using the Kullback-Leibler divergence as the objective function.
• Applies optimization techniques in the space of probability measures equipped with the Wasserstein distance to derive the descent lemma.
• Uses a kernel integral operator to smooth the gradient direction, restricting descent to the unit ball of a Reproducing Kernel Hilbert Space (RKHS).
• Derives a propagation of chaos bound via techniques from Jourdain et al. (2007), relating the empirical particle distribution to its population-level counterpart.
• Imposes regularity conditions on the potential function $V$ and kernel $k$, including Lipschitz continuity and boundedness, to ensure stability and convergence.
• Analyzes both continuous-time dynamics and discrete-time iterations of SVGD, with explicit bounds on the Wasserstein distance between the finite-particle and population distributions.

Experimental results

Research questions

• RQ1 Can a non-asymptotic descent lemma be established for SVGD in the infinite-particle regime with a constant step-size?
• RQ2 What are the convergence rates of the average Stein Fisher divergence (Kernel Stein Discrepancy) in the population limit?
• RQ3 How does the empirical distribution of finite particles in SVGD deviate from its population-level counterpart over time?
• RQ4 Can a uniform-in-time propagation of chaos bound be derived for the SVGD particle system?
• RQ5 Under what conditions does the finite-particle SVGD system converge to the target distribution $\pi$ as both the number of particles $N$ and iterations $n$ increase?

Key findings

• A descent lemma is established for SVGD in the infinite-particle regime, showing that the algorithm decreases the objective at each iteration with a constant step-size.
• Convergence rates are provided for the average Stein Fisher divergence (Kernel Stein Discrepancy), quantifying the decay of discrepancy to zero over time.
• A propagation of chaos bound is derived, showing that $\mathbb{E}[W_2^2(\mu_n, \hat{\mu}_n)] \leq \frac{1}{2}\left(\frac{1}{\sqrt{N}}\sqrt{\text{var}(\mu_0)}e^{LT}\right)(e^{2LT}-1)$, where $L$ depends on the kernel and target distribution.
• The bound depends on the number of particles $N$, the time horizon $T$, and the initial variance, but not on uniform-in-time decay, indicating a non-uniform propagation of chaos.
• The results are non-asymptotic and explicitly quantify the trade-off between particle count, iteration count, and approximation error.
• The analysis identifies open problems, including deriving uniform-in-time propagation of chaos and convergence rates in terms of KL divergence under convexity or log-Sobolev conditions.