[Paper Review] Stein Variational Gradient Descent: many-particle and long-time asymptotics

Nikolas Nüsken, D. R. Michiel Renger|arXiv (Cornell University)|Feb 25, 2021

Markov Chains and Monte Carlo Methods4 citations

TL;DR

This paper establishes a rigorous connection between variational inference and Markov chain Monte Carlo through Stein variational gradient descent (SVGD), showing that SVGD's deterministic and stochastic variants arise from gradient flows in a Stein geometry. It proves that in the long-time and many-particle limit, the kernelised Stein discrepancy (Stein-Fisher information) governs the large-deviation rate functional, providing a variational characterization of SVGD's convergence behavior.

ABSTRACT

Stein variational gradient descent (SVGD) refers to a class of methods for Bayesian inference based on interacting particle systems. In this paper, we consider the originally proposed deterministic dynamics as well as a stochastic variant, each of which represent one of the two main paradigms in Bayesian computational statistics: variational inference and Markov chain Monte Carlo. As it turns out, these are tightly linked through a correspondence between gradient flow structures and large-deviation principles rooted in statistical physics. To expose this relationship, we develop the cotangent space construction for the Stein geometry, prove its basic properties, and determine the large-deviation functional governing the many-particle limit for the empirical measure. Moreover, we identify the Stein-Fisher information (or kernelised Stein discrepancy) as its leading order contribution in the long-time and many-particle regime in the sense of $Γ$-convergence, shedding some light on the finite-particle properties of SVGD. Finally, we establish a comparison principle between the Stein-Fisher information and RKHS-norms that might be of independent interest.

Motivation & Objective

• To unify variational inference and Markov chain Monte Carlo by analyzing Stein variational gradient descent (SVGD) as a unified framework.
• To establish a correspondence between gradient flow structures and large-deviation principles in the context of SVGD.
• To characterize the long-time and many-particle asymptotics of SVGD using large-deviation theory.
• To identify the kernelised Stein discrepancy (Stein-Fisher information) as the leading-order contribution in the $γ$-convergence limit of the large-deviation functional.
• To develop a cotangent space construction for Stein geometry and prove its basic properties for the analysis of SVGD dynamics.

Proposed method

• Develops a cotangent space construction for the Stein geometry to analyze the Riemannian structure underlying SVGD.
• Derives the large-deviation functional governing the empirical measure in the many-particle limit of SVGD.
• Uses $γ$-convergence to show that the leading-order term in the large-deviation rate functional is the kernelised Stein discrepancy (Stein-Fisher information).
• Analyzes both the deterministic and stochastic variants of SVGD as gradient flows in the Stein geometry, linking them to variational inference and MCMC paradigms.
• Applies Itô's lemma to the empirical measure dynamics to derive the generator of the stochastic SVGD process and derive the associated Fokker-Planck-type equation.
• Establishes a comparison principle between the Stein-Fisher information and RKHS-norms, revealing structural insights into the geometry of the problem.

Experimental results

Research questions

• RQ1 How are the deterministic and stochastic variants of SVGD related through gradient flow and large-deviation principles?
• RQ2 What is the large-deviation rate functional that governs the empirical measure in the many-particle limit of SVGD?
• RQ3 How does the kernelised Stein discrepancy (Stein-Fisher information) emerge as the leading-order term in the long-time and many-particle regime?
• RQ4 What is the role of the cotangent space in the Stein geometry, and how does it support the analysis of SVGD dynamics?
• RQ5 How do the Stein-Fisher information and RKHS-norms compare in terms of their geometric and variational properties?

Key findings

• The large-deviation functional for the empirical measure in the many-particle SVGD limit is characterized, with the kernelised Stein discrepancy as its leading-order term in the $γ$-convergence sense.
• The deterministic SVGD dynamics correspond to a gradient flow in the Stein geometry, with the $γ$-limit of the large-deviation rate functional being the Stein-Fisher information.
• The stochastic SVGD variant is shown to correspond to a diffusion process whose invariant measure is the target distribution, linking it to MCMC methods.
• A comparison principle is established between the Stein-Fisher information and RKHS-norms, showing that the former dominates the latter in a certain variational sense.
• The cotangent space construction for the Stein geometry is rigorously developed and shown to support the analysis of SVGD's gradient flow structure.
• The paper confirms that SVGD's convergence behavior in the long-time and many-particle regime is governed by the kernelised Stein discrepancy, providing a theoretical foundation for its use in Bayesian inference.