[Paper Review] SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator

Cong Fang, Chris Junchi Li+2 more|arXiv (Cornell University)|Jul 4, 2018

Stochastic Gradient Optimization TechniquesComputer Science38 references74 citations

TL;DR

SPIDER introduces a stochastic path-integrated differential estimator to track deterministic quantities with significantly reduced sampling costs, yielding near-optimal rates for non-convex stochastic optimization in first- and second-order settings, plus a zeroth-order variant.

ABSTRACT

In this paper, we propose a new technique named extit{Stochastic Path-Integrated Differential EstimatoR} (SPIDER), which can be used to track many deterministic quantities of interest with significantly reduced computational cost. We apply SPIDER to two tasks, namely the stochastic first-order and zeroth-order methods. For stochastic first-order method, combining SPIDER with normalized gradient descent, we propose two new algorithms, namely SPIDER-SFO and SPIDER-SFO extsuperscript{+}, that solve non-convex stochastic optimization problems using stochastic gradients only. We provide sharp error-bound results on their convergence rates. In special, we prove that the SPIDER-SFO and SPIDER-SFO extsuperscript{+} algorithms achieve a record-breaking gradient computation cost of $\mathcal{O}\left( \min( n^{1/2} ε^{-2}, ε^{-3} ) ight)$ for finding an $ε$-approximate first-order and $ ilde{\mathcal{O}}\left( \min( n^{1/2} ε^{-2}+ε^{-2.5}, ε^{-3} ) ight)$ for finding an $(ε, \mathcal{O}(ε^{0.5}))$-approximate second-order stationary point, respectively. In addition, we prove that SPIDER-SFO nearly matches the algorithmic lower bound for finding approximate first-order stationary points under the gradient Lipschitz assumption in the finite-sum setting. For stochastic zeroth-order method, we prove a cost of $\mathcal{O}( d \min( n^{1/2} ε^{-2}, ε^{-3}) )$ which outperforms all existing results.

Motivation & Objective

• Motivate and address non-convex stochastic optimization efficiently using stochastic gradients only.
• Develop a new estimator, SPIDER, to track deterministic quantities with reduced sampling costs.
• Enable faster convergence rates for finding approximate first- and second-order stationary points.
• Extend SPIDER to zeroth-order optimization and show improved function-evaluation costs.

Proposed method

• Propose Stochastic Path-Integrated Differential Estimator (SPIDER) to track quantities like gradients with lower sampling costs.
• Combine SPIDER with Normalized Gradient Descent (NGD) to create SPIDER-SFO and SPIDER-SFO+ for non-convex optimization.
• Derive error bounds showing SPIDER-based estimators maintain controlled variance and bias (martingale-based analysis).
• Apply SPIDER to stochastic zeroth-order methods and obtain reduced function-value access costs.
• Provide convergence theorems for finding ε-approximate first-order points and (ε, ε^0.5)-approximate second-order points, including finite-sum and online settings.

Experimental results

Research questions

• RQ1 Can SPIDER reduce the gradient-sampling complexity needed to find ε-approximate first-order stationary points in non-convex stochastic optimization?
• RQ2 Can SPIDER achieve near-optimal rates for finding ε-approximate second-order stationary points under standard smoothness assumptions?
• RQ3 What are the benefits and costs of applying SPIDER to zeroth-order non-convex optimization?
• RQ4 How does SPIDER compare to existing variance-reduction and saddle-point-escaping methods in terms of gradient complexity and robustness?

Key findings

• SPIDER-SFO achieves a gradient computation cost of O(min(n^1/2 ε^-2, ε^-3)) for finding an ε-approximate first-order stationary point.
• SPIDER-SFO+ (with Negative-Curvature-Search) achieves a gradient cost of Õ(min(n^1/2 ε^-2 + ε^-2.5, ε^-3)) for an (ε, O(ε^0.5))-approximate second-order stationary point under Hessian-Lipschitzity.
• In the online/finite-sum setting, SPIDER nearly matches the algorithmic lower bound for finding approximate first-order stationary points, up to poly-log factors and constants.
• SPIDER for zeroth-order optimization achieves a cost of O(d min(n^1/2 ε^-2, ε^-3)) function evaluations, improving upon existing results.
• The analysis provides a simpler convergence framework that can be extended to other algorithms such as SGD, SVRG, and SAGA.