[Paper Review] On Stochastic Subgradient Mirror-Descent Algorithm with Weighted Averaging

Angelia Nedich, Soomin Lee|arXiv (Cornell University)|Jul 7, 2013

Stochastic Gradient Optimization Techniques19 references4 citations

TL;DR

This paper proposes a stochastic subgradient mirror-descent algorithm with weighted iterate averaging that achieves optimal convergence rates—$1/k$ for strongly convex functions and $1/\sqrt{k}$ for general convex functions—using simpler update rules than prior methods. The key innovation lies in stepsize and weight selection inspired by Tseng's acceleration framework, enabling almost sure subsequence convergence with non-summable stepsizes like $1/\sqrt{k+1}$, a novel result in the literature.

ABSTRACT

This paper considers stochastic subgradient mirror-descent method for solving constrained convex minimization problems. In particular, a stochastic subgradient mirror-descent method with weighted iterate-averaging is investigated and its per-iterate convergence rate is analyzed. The novel part of the approach is in the choice of weights that are used to construct the averages. Through the use of these weighted averages, we show that the known optimal rates can be obtained with simpler algorithms than those currently existing in the literature. Specifically, by suitably choosing the stepsize values, one can obtain the rate of the order $1/k$ for strongly convex functions, and the rate $1/\sqrt{k}$ for general convex functions (not necessarily differentiable). Furthermore, for the latter case, it is shown that a stochastic subgradient mirror-descent with iterate averaging converges (along a subsequence) to an optimal solution, almost surely, even with the stepsize of the form $1/\sqrt{1+k}$, which was not previously known. The stepsize choices that achieve the best rates are those proposed by Paul Tseng for acceleration of proximal gradient methods.

Motivation & Objective

• To develop a simpler stochastic subgradient mirror-descent algorithm that achieves optimal convergence rates for convex and strongly convex problems.
• To establish almost sure subsequence convergence for the averaged iterates under non-summable stepsizes, particularly $\alpha_k = 1/\sqrt{k+1}$, which was previously unknown.
• To unify and simplify existing accelerated methods by using a novel Lyapunov function and stepsize-weighting strategy inspired by Tseng and Nesterov.
• To demonstrate through simulations that the proposed method matches or exceeds the performance of existing accelerated algorithms like AC-SA and Nesterov's PD method.

Proposed method

• Uses a stochastic subgradient mirror-descent update with a Bregman distance-based projection onto a convex constraint set.
• Applies weighted averaging of iterates using weights derived from the stepsize sequence, specifically $\alpha_k = 1/\sqrt{k+1}$ or $\alpha_k = 1/k$ for different cases.
• Introduces a novel Lyapunov function to analyze convergence, combining the mirror-descent progress with the averaging dynamics.
• Employs stepsize rules inspired by Tseng’s acceleration framework, which are shown to yield optimal rates without requiring complex three-sequence constructions.
• Derives convergence rates via a unified analysis that applies to both strongly convex and general convex functions with compact constraint sets.
• Uses the averaged iterate $\hat{x}_k = \sum_{i=1}^k w_i x_i / \sum_{i=1}^k w_i$ as the final output, where weights $w_i$ are proportional to the stepsize sequence.

Experimental results

Research questions

• RQ1 Can a simpler stochastic mirror-descent algorithm achieve the optimal $1/k$ convergence rate for strongly convex functions without complex three-sequence updates?
• RQ2 Does weighted averaging with non-summable stepsizes like $1/\sqrt{k+1}$ guarantee almost sure subsequence convergence to the optimal solution?
• RQ3 Can the proposed method match or exceed the performance of existing accelerated stochastic subgradient methods like AC-SA and Nesterov’s PD method?
• RQ4 What is the role of the Lyapunov function in enabling a unified convergence analysis for both strongly convex and general convex cases?

Key findings

• The proposed algorithm achieves the optimal $1/k$ convergence rate for strongly convex functions using a simpler update rule than prior accelerated methods such as those in Ghadimi and Lan [9] and Juditsky et al. [10].
• For general convex functions with compact constraint sets, the method achieves the best-known $1/\sqrt{k}$ convergence rate using a stepsize of $\alpha_k = 1/\sqrt{k+1}$.
• The paper establishes the first almost sure subsequence convergence result for stochastic mirror-descent with a non-summable stepsize $\alpha_k = 1/\sqrt{k+1}$, a significant theoretical advance.
• Simulations show that the method performs comparably to AC-SA and Nesterov’s PD method, with slight advantages in early iterations for some instances.
• The weighted averaging scheme, with weights derived from the stepsize, enables optimal rates without requiring additional memory or complex sequence tracking.
• The use of a novel Lyapunov function simplifies the convergence analysis and unifies the treatment of both strongly convex and general convex cases.