[Paper Review] High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise

Eduard Gorbunov, Abdurakhmon Sadiev+6 more|arXiv (Cornell University)|Oct 3, 2023

Stochastic Gradient Optimization TechniquesComputer Science3 citations

TL;DR

This paper proposes novel stochastic optimization methods for composite and distributed problems with heavy-tailed noise, using gradient difference clipping to achieve tight high-probability convergence guarantees. It establishes nearly optimal convergence rates, including linear convergence for strongly convex problems, resolving limitations in prior work that failed to handle composite or distributed settings under heavy-tailed noise.

ABSTRACT

High-probability analysis of stochastic first-order optimization methods under mild assumptions on the noise has been gaining a lot of attention in recent years. Typically, gradient clipping is one of the key algorithmic ingredients to derive good high-probability guarantees when the noise is heavy-tailed. However, if implemented naïvely, clipping can spoil the convergence of the popular methods for composite and distributed optimization (Prox-SGD/Parallel SGD) even in the absence of any noise. Due to this reason, many works on high-probability analysis consider only unconstrained non-distributed problems, and the existing results for composite/distributed problems do not include some important special cases (like strongly convex problems) and are not optimal. To address this issue, we propose new stochastic methods for composite and distributed optimization based on the clipping of stochastic gradient differences and prove tight high-probability convergence results (including nearly optimal ones) for the new methods. Using similar ideas, we also develop new methods for composite and distributed variational inequalities and analyze the high-probability convergence of these methods.

Motivation & Objective

• To address the lack of high-probability convergence analysis for composite and distributed stochastic optimization under heavy-tailed noise.
• To overcome the failure of standard gradient clipping in proximal and distributed methods, even without noise.
• To develop new algorithms that achieve accelerated and linear convergence rates with high probability in the presence of heavy-tailed noise.
• To extend the analysis to composite and distributed variational inequalities under similar noise conditions.

Proposed method

• Proposes a new clipping mechanism based on stochastic gradient differences rather than raw gradients to stabilize convergence in composite and distributed settings.
• Introduces Prox-clipped-SGD and Prox-clipped-SGD-shift variants that use adaptive clipping of gradient differences to maintain convergence under heavy-tailed noise.
• Employs a high-probability analysis framework using concentration inequalities and event decomposition to bound deviation probabilities.
• Derives tight complexity bounds by controlling the growth of gradient differences through clipping and stepsize scheduling.
• Applies the method to both minimization and variational inequality problems, extending to distributed and composite formulations.
• Uses a recursive event-based analysis to show that the iterates converge to the optimal solution with high probability, even under heavy-tailed noise.

Experimental results

Research questions

• RQ1 Can gradient difference clipping enable high-probability convergence in composite and distributed stochastic optimization under heavy-tailed noise?
• RQ2 Why do standard gradient clipping methods fail in proximal and distributed settings, even in the noiseless case?
• RQ3 Can nearly optimal convergence rates (including linear convergence) be achieved for strongly convex problems under heavy-tailed noise using new clipping strategies?
• RQ4 Can the proposed methods be extended to variational inequalities with constraints and composite structures?
• RQ5 What is the optimal trade-off between clipping level, stepsize, and convergence rate in high-probability regimes?

Key findings

• The proposed Prox-clipped-SGD and Prox-clipped-SGD-shift methods achieve high-probability convergence with nearly optimal rates for composite and distributed problems under heavy-tailed noise.
• The method ensures linear convergence for strongly convex problems with high probability, overcoming limitations of prior approaches.
• The analysis shows that with probability at least 1−β, the iterate satisfies ∥xK+1−x∗∥2≤O(Vexp(−Kln2(nK)/β)), achieving nearly optimal dependence on problem parameters.
• For the strongly convex case, the method achieves a convergence rate of O(exp(−μK/L)) with high probability, matching the optimal in-expectation rate.
• The complexity bound for achieving ε-accuracy is K=O(max(L/√nμ, L/μ ln(V/ε), (σ²/μ²ε)^(α/(2(α−1))) ln(n/β))), which is tight up to logarithmic factors.
• Numerical experiments confirm that gradient difference clipping outperforms naive gradient clipping, especially with small clipping levels, and enables faster convergence to higher accuracy.