[Paper Review] On the Convergence of Stochastic Gradient MCMC Algorithms with High-Order Integrators

Changyou Chen, Nan Ding+1 more|arXiv (Cornell University)|Oct 21, 2016

Markov Chains and Monte Carlo Methods25 references124 citations

TL;DR

The paper develops weak convergence theory for SG-MCMC methods with high-order integrators and shows that a 2nd-order symmetric splitting integrator improves convergence (e.g., SGHMC achieves MSE rate L^{-4/5} vs L^{-2/3} with Euler).

ABSTRACT

Recent advances in Bayesian learning with large-scale data have witnessed emergence of stochastic gradient MCMC algorithms (SG-MCMC), such as stochastic gradient Langevin dynamics (SGLD), stochastic gradient Hamiltonian MCMC (SGHMC), and the stochastic gradient thermostat. While finite-time convergence properties of the SGLD with a 1st-order Euler integrator have recently been studied, corresponding theory for general SG-MCMCs has not been explored. In this paper we consider general SG-MCMCs with high-order integrators, and develop theory to analyze finite-time convergence properties and their asymptotic invariant measures. Our theoretical results show faster convergence rates and more accurate invariant measures for SG-MCMCs with higher-order integrators. For example, with the proposed efficient 2nd-order symmetric splitting integrator, the {\em mean square error} (MSE) of the posterior average for the SGHMC achieves an optimal convergence rate of $L^{-4/5}$ at $L$ iterations, compared to $L^{-2/3}$ for the SGHMC and SGLD with 1st-order Euler integrators. Furthermore, convergence results of decreasing-step-size SG-MCMCs are also developed, with the same convergence rates as their fixed-step-size counterparts for a specific decreasing sequence. Experiments on both synthetic and real datasets verify our theory, and show advantages of the proposed method in two large-scale real applications.

Motivation & Objective

• Develop weak convergence theory for general SG-MCMC with high-order integrators.
• Characterize finite-time bias and MSE for K-th order integrators under fixed and decreasing step sizes.
• Introduce a numerically efficient 2nd-order symmetric splitting integrator for SG-MCMC.
• Analyze impact of stochastic gradient noise on convergence and invariant measures.

Proposed method

• Model SG-MCMC as Itô diffusions with generator L to study weak convergence of expectations of smooth statistics.
• Use Poisson equation to relate posterior averages to a solution psi and derive bias/MSE bounds.
• Introduce K-th order local integrators with P_h approximating e^{hL} and extend to stochastic-gradient settings with tilde{L}_l.
• Derive bounds: bias = O(1/(Lh) + sum_l E||E Delta V_l|| / L + h^K) and MSE = O( (1/L) sum_l E||Delta V_l||^2 / L + 1/(Lh) + h^{2K} ).
• Propose and analyze a 2nd-order symmetric splitting integrator (ABOBA) for SGHMC and prove it is a 2nd-order local integrator.

Experimental results

Research questions

• RQ1 How does the order K of the numerical integrator affect the finite-time bias and MSE of SG-MCMC algorithms?
• RQ2 What are the convergence rates for fixed-step-size SG-MCMC with high-order integrators, and how do they compare to 1st-order Euler schemes?
• RQ3 How do stochastic gradient noise and step-size schedules (fixed vs decreasing) influence the asymptotic invariant measures and convergence guarantees?
• RQ4 Can a practical high-order integrator (e.g., 2nd-order symmetric splitting) improve performance for large-scale Bayesian learning (SGHMC/SGLD) in real data?

Key findings

• For a K-th order integrator, the bias at iteration L is O(1/(Lh) + sum_l E||E Delta V_l||/L + h^K).
• The MSE at iteration L is O( (1/L) sum_l E||Delta V_l||^2 / L + 1/(Lh) + h^{2K} ).
• With a 2nd-order symmetric splitting integrator (K=2), SGHMC achieves a faster optimal bias rate L^{-2/3} and MSE rate L^{-4/5} (with h proportional to L^{-1/5}) compared to Euler-based L^{-1/2} bias and L^{-2/3} MSE for SGLD/SGHMC.
• Invariant measures of SG-MCMCs converge to the true posterior measure with distance d( ilde{ ho}_h, ho) = O(h^K) (K-th order integrator).
• Decreasing-step-size SG-MCMCs are consistent; for h_l ~ l^{-α}, optimal α values align with fixed-step results (α=1/(K+1) for bias, α=1/(2K+1) for MSE).
• Experiments on synthetic and large-scale data (LDA, SBN/MNIST) show the splitting-based SGHMC (SGHMC-S) outperforms Euler-based methods and avoids instability seen with large step sizes.