[Paper Review] On Almost Sure Convergence Rates of Stochastic Gradient Methods

Jun Liu, Ye Yuan|arXiv (Cornell University)|Feb 9, 2022

Stochastic Gradient Optimization Techniques4 citations

TL;DR

This paper provides a unified almost sure convergence rate analysis for stochastic gradient descent (SGD), stochastic heavy-ball (SHB), and stochastic Nesterov’s accelerated gradient (SNAG) methods. It establishes that for strongly convex functions, the almost sure convergence rates are arbitrarily close to optimal; for non-convex functions, it proves almost sure convergence of both the last iterates and a weighted average of squared gradient norms; and for general convex functions, it derives the first last-iterate almost sure convergence rates, though not yet optimal.

ABSTRACT

The vast majority of convergence rates analysis for stochastic gradient methods in the literature focus on convergence in expectation, whereas trajectory-wise almost sure convergence is clearly important to ensure that any instantiation of the stochastic algorithms would converge with probability one. Here we provide a unified almost sure convergence rates analysis for stochastic gradient descent (SGD), stochastic heavy-ball (SHB), and stochastic Nesterov's accelerated gradient (SNAG) methods. We show, for the first time, that the almost sure convergence rates obtained for these stochastic gradient methods on strongly convex functions, are arbitrarily close to their optimal convergence rates possible. For non-convex objective functions, we not only show that a weighted average of the squared gradient norms converges to zero almost surely, but also the last iterates of the algorithms. We further provide last-iterate almost sure convergence rates analysis for stochastic gradient methods on weakly convex smooth functions, in contrast with most existing results in the literature that only provide convergence in expectation for a weighted average of the iterates.

Motivation & Objective

• To address the lack of almost sure convergence rate analysis for stochastic heavy-ball (SHB) and stochastic Nesterov’s accelerated gradient (SNAG) methods in non-convex and strongly convex settings.
• To provide a unified theoretical framework for analyzing almost sure convergence rates across SGD, SHB, and SNAG.
• To establish last-iterate almost sure convergence rates for general convex functions, a gap not previously filled in the literature.
• To show that convergence rates for strongly convex functions are arbitrarily close to the optimal possible rates under almost sure convergence.

Proposed method

• Uses a unified analysis framework based on martingale convergence and almost sure convergence theorems to analyze SGD, SHB, and SNAG.
• Applies the law of large numbers for martingales and boundedness in L² to prove almost sure convergence of gradient-related terms.
• Employs step size rules of the form $\alpha_t = \Theta(1/t^{2/3 + \varepsilon})$ for $\varepsilon \in (0, 1/3)$ to derive last-iterate convergence rates.
• Leverages the boundedness of $\sum \|v_t\|^2$ and $\sum \alpha_t w_t$ to show almost sure convergence of gradients and iterates.
• Uses a decomposition of stochastic gradients into bias and variance terms: $\alpha_t w_t = \alpha_t(g_t - \nabla f(x_t)) + \alpha_t(\nabla f(x_t) - \nabla f(z_t))$, and proves almost sure convergence of each component.
• Applies Lemma 4.1 with $b_t = \|\nabla f(z_t)\|$ and $p=2$ to show $\nabla f(z_t) \to 0$ almost surely.

Experimental results

Research questions

• RQ1 Can almost sure convergence rates be established for SHB and SNAG on strongly convex functions, and how close are they to the optimal rates?
• RQ2 For non-convex smooth functions, do the last iterates of SHB and SNAG converge almost surely, and what is the rate of convergence of the gradient norms?
• RQ3 Is it possible to derive last-iterate almost sure convergence rates for SGD, SHB, and SNAG on general convex functions, and how do they compare to existing expectation-based rates?
• RQ4 Can the almost sure convergence rates for these methods be made arbitrarily close to the optimal rates in the strongly convex case?
• RQ5 What is the sharpest possible almost sure convergence rate for last iterates in the general convex case, and can it be improved beyond $O(1/t^{1/3 - \varepsilon})$?

Key findings

• For strongly convex functions, the almost sure convergence rates of SGD, SHB, and SNAG are arbitrarily close to the optimal convergence rates possible.
• For non-convex smooth functions, the weighted average of the squared gradient norms converges to zero almost surely, and the last iterates of SHB and SNAG converge almost surely to a minimizer.
• For general convex functions, the paper establishes the first last-iterate almost sure convergence rates of $O(1/t^{1/3 - \varepsilon})$ for SGD, SHB, and SNAG.
• The convergence rate $O(1/t^{1/3 - \varepsilon})$ for general convex functions is not yet optimal, as the lower bound for convergence in expectation is tighter.
• The analysis shows that $\sum \|v_t\|^2 < \infty$ almost surely, which implies $v_t \to 0$ almost surely, and $\nabla f(x_t) \to 0$ almost surely.
• The proof technique relies on decomposing the stochastic gradient error into a martingale difference and a bias term, both of which are shown to converge almost surely under the chosen step size rule.