[Paper Review] Linear Convergence of Variance-Reduced Projected Stochastic Gradient without Strong Convexity.

Pinghua Gong, Jieping Ye|arXiv (Cornell University)|Jun 4, 2014

Stochastic Gradient Optimization Techniques12 citations

TL;DR

This paper introduces Variance-Reduced Projected Stochastic Gradient (VRPSG) and establishes linear convergence for non-strongly convex optimization problems, a class common in machine learning. By proving a novel Semi-Strongly Convex (SSC) inequality, the authors show that VRPSG and Prox-SVRG achieve linear convergence without requiring strong convexity, marking the first such result for both constrained and regularized settings.

ABSTRACT

Stochastic gradient algorithms estimate the gradient based on only one or a few samples and enjoy low computational cost per iteration. They have been widely used in large-scale optimization problems. However, stochastic gradient algorithms are usually slow to converge and achieve sub-linear convergence rates, due to the inherent variance in the gradient computation. To accelerate the convergence, some variance-reduced stochastic gradient algorithms, e.g., proximal stochastic variance-reduced gradient (Prox-SVRG) algorithm, have recently been proposed to solve strongly convex problems. Under the strongly convex condition, these variance-reduced stochastic gradient algorithms achieve a linear convergence rate. However, many machine learning problems are convex but not strongly convex. In this paper, we introduce Prox-SVRG and its projected variant called Variance-Reduced Projected Stochastic Gradient (VRPSG) to solve a class of non-strongly convex optimization problems widely used in machine learning. As the main technical contribution of this paper, we show that both VRPSG and Prox-SVRG achieve a linear convergence rate without strong convexity. A key ingredient in our proof is a Semi-Strongly Convex (SSC) inequality which is the first to be rigorously proved for a class of non-strongly convex problems in both constrained and regularized settings. Moreover, the SSC inequality is independent of algorithms and may be applied to analyze other stochastic gradient algorithms besides VRPSG and Prox-SVRG, which may be of independent interest. To the best of our knowledge, this is the first work that establishes the linear convergence rate for the variance-reduced stochastic gradient algorithms on solving both constrained and regularized problems without strong convexity.

Motivation & Objective

• To address the limitation of existing variance-reduced stochastic gradient methods, which require strong convexity to achieve linear convergence.
• To extend linear convergence guarantees to non-strongly convex problems, which are prevalent in machine learning applications.
• To develop a theoretical framework that supports linear convergence in constrained and regularized optimization without strong convexity.
• To introduce and rigorously prove the Semi-Strongly Convex (SSC) inequality as a key analytical tool for non-strongly convex problems.

Proposed method

• Proposes a projected variant of Prox-SVRG, named Variance-Reduced Projected Stochastic Gradient (VRPSG), for solving constrained optimization problems.
• Introduces a new theoretical condition called Semi-Strongly Convex (SSC) inequality, which holds for a class of non-strongly convex problems.
• Establishes that the SSC inequality enables linear convergence of VRPSG and Prox-SVRG even without strong convexity.
• Uses the SSC inequality as a core analytical tool to bound the progress of the algorithm per iteration, independent of the algorithm design.
• Applies the SSC condition to both regularized and constrained optimization settings, demonstrating broad applicability.
• Proves that the convergence rate remains linear by showing that the expected suboptimality decays exponentially under the SSC condition.

Experimental results

Research questions

• RQ1 Can variance-reduced stochastic gradient methods achieve linear convergence for non-strongly convex problems?
• RQ2 What theoretical condition enables linear convergence in the absence of strong convexity?
• RQ3 Is the Semi-Strongly Convex (SSC) inequality a valid and generalizable condition for non-strongly convex problems?
• RQ4 Can the SSC inequality be used to analyze other stochastic gradient algorithms beyond VRPSG and Prox-SVRG?
• RQ5 Does the proposed VRPSG method maintain linear convergence in both constrained and regularized settings without strong convexity?

Key findings

• VRPSG achieves linear convergence for non-strongly convex problems, a result previously unproven for this class of problems.
• The proposed Semi-Strongly Convex (SSC) inequality is rigorously proven and holds for a broad class of non-strongly convex problems.
• The SSC inequality is independent of the optimization algorithm and can be applied to analyze other stochastic gradient methods.
• The linear convergence rate is established for both constrained and regularized optimization problems without requiring strong convexity.
• This work is the first to demonstrate linear convergence of variance-reduced stochastic gradient algorithms in non-strongly convex settings.
• The theoretical framework based on SSC provides a new pathway for analyzing convergence in large-scale machine learning problems.