[Paper Review] On stochastic gradient Langevin dynamics with dependent data streams: the fully non-convex case

Ngọc Huy Châu, Éric Moulines+3 more|Edinburgh Research Explorer|May 30, 2019

Markov Chains and Monte Carlo Methods33 references26 citations

TL;DR

This paper establishes non-asymptotic convergence guarantees for Stochastic Gradient Langevin Dynamics (SGLD) in the fully non-convex setting with dependent data streams, using the $L^1$-Wasserstein distance. By comparing SGLD to an auxiliary diffusion process and leveraging contraction estimates, it achieves sharper, uniform convergence rates in terms of stepsize, extending prior results beyond i.i.d. data and log-concave targets.

ABSTRACT

We consider the problem of sampling from a target distribution, which is \emph {not necessarily logconcave}, in the context of empirical risk minimization and stochastic optimization as presented in Raginsky et al. (2017). Non-asymptotic analysis results are established in the $L^1$-Wasserstein distance for the behaviour of Stochastic Gradient Langevin Dynamics (SGLD) algorithms. We allow the estimation of gradients to be performed even in the presence of \emph{dependent} data streams. Our convergence estimates are sharper and \emph{uniform} in the number of iterations, in contrast to those in previous studies.

Motivation & Objective

• To provide non-asymptotic convergence rates for SGLD in the fully non-convex case with dependent data streams.
• To extend existing convergence guarantees beyond i.i.d. data and log-concave target distributions.
• To improve upon prior $L^2$-Wasserstein bounds by using the $L^1$-Wasserstein metric for sharper, uniform estimates.
• To establish convergence under a dissipativity condition on the potential function $U$, without requiring log-concavity.

Proposed method

• The authors compare the discrete SGLD process to a continuous-time auxiliary diffusion process inspired by the overdamped Langevin SDE.
• They employ contraction estimates for diffusions from [18] to bound the distance between the SGLD and the target distribution.
• A coupling-based approach is used to relate the $L^1$-Wasserstein distance to the Kullback-Leibler divergence via weighted Pinsker-type inequalities.
• The analysis relies on a measurable function $V$ to control moments and ensure integrability in the $V$-norm.
• Key technical tools include Girsanov’s theorem for likelihood ratio computation and moment bounds for the SDE solution.
• The method allows for gradient estimates from dependent data streams by assuming a dissipativity condition on $U$.

Experimental results

Research questions

• RQ1 Can non-asymptotic convergence rates for SGLD be established in the fully non-convex case with dependent data streams?
• RQ2 Does using the $L^1$-Wasserstein distance yield sharper convergence bounds than previous $L^2$-Wasserstein estimates?
• RQ3 Can contraction techniques for diffusions be adapted to analyze discrete SGLD algorithms under general dissipativity conditions?
• RQ4 How do the convergence rates scale with stepsize and iteration count in the absence of log-concavity?
• RQ5 What is the role of the $V$-norm and coupling in bounding the Wasserstein distance for non-log-concave targets?

Key findings

• The paper establishes non-asymptotic convergence in the $L^1$-Wasserstein distance for SGLD under a dissipativity condition, even with dependent data streams.
• The convergence rates are sharper and uniform in the number of iterations compared to prior $L^2$-Wasserstein bounds.
• The $L^1$-Wasserstein distance is bounded using a weighted Pinsker inequality that relates it to the Kullback-Leibler divergence of the laws.
• The analysis shows that the convergence rate depends on the stepsize and the Lipschitz constant of $\nabla U$, with explicit dependence derived via coupling and Girsanov's theorem.
• The method achieves uniform bounds across iterations, avoiding the degradation seen in some prior analyses.
• The results extend the applicability of SGLD to non-i.i.d. and non-log-concave settings, providing stronger theoretical guarantees for optimization in big data and online learning.