[Paper Review] Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization

Xiao Wang, Shiqian Ma+2 more|arXiv (Cornell University)|Jul 5, 2016

Stochastic Gradient Optimization Techniques45 references16 citations

TL;DR

This paper proposes a stochastic damped L-BFGS (SdLBFGS) method for nonconvex stochastic optimization, leveraging a general framework for stochastic quasi-Newton methods with almost sure convergence to stationary points. It achieves an O(ε⁻²) SFO-calls complexity for finding ε-approximate stationary points and introduces a variance-reduced variant (SdLBFGS-VR) that outperforms SVRG and standard SdLBFGS on SVM and neural network tasks.

ABSTRACT

In this paper we study stochastic quasi-Newton methods for nonconvex stochastic optimization, where we assume that noisy information about the gradients of the objective function is available via a stochastic first-order oracle (SFO). We propose a general framework for such methods, for which we prove almost sure convergence to stationary points and analyze its worst-case iteration complexity. When a randomly chosen iterate is returned as the output of such an algorithm, we prove that in the worst-case, the SFO-calls complexity is $O(ε^{-2})$ to ensure that the expectation of the squared norm of the gradient is smaller than the given accuracy tolerance $ε$. We also propose a specific algorithm, namely a stochastic damped L-BFGS (SdLBFGS) method, that falls under the proposed framework. {Moreover, we incorporate the SVRG variance reduction technique into the proposed SdLBFGS method, and analyze its SFO-calls complexity. Numerical results on a nonconvex binary classification problem using SVM, and a multiclass classification problem using neural networks are reported.

Motivation & Objective

• Address the challenge of nonconvex stochastic optimization where only noisy gradient estimates are available via a stochastic first-order oracle (SFO).
• Develop a general framework for stochastic quasi-Newton methods that ensures global convergence to stationary points under mild conditions.
• Design a damped L-BFGS variant (SdLBFGS) that maintains positive definiteness of the Hessian approximation without requiring convexity.
• Integrate variance reduction (SVRG) into the SdLBFGS framework to improve convergence speed and stability.
• Demonstrate the effectiveness of the proposed methods on nonconvex classification problems using SVM and deep neural networks.

Proposed method

• Propose a general stochastic quasi-Newton framework where search directions are computed using a damped BFGS update to preserve positive definiteness of the Hessian approximation.
• Use a stochastic first-order oracle (SFO) to obtain noisy gradient estimates, avoiding full gradient computation at each iteration.
• Incorporate a damping technique to ensure the Hessian approximation remains positive definite even when the curvature condition $ s_{k-1}^T y_{k-1} < 0 $ is violated.
• Introduce SdLBFGS-VR by combining the SdLBFGS method with the SVRG variance reduction technique, using a full gradient computed periodically.
• Apply a constant step size in SdLBFGS-VR and a diminishing step size in SdLBFGS to ensure convergence, with the outer loop structure of SVRG used to reduce gradient variance.
• Use a randomly selected iterate as the output, with theoretical analysis of worst-case SFO-calls complexity.

Experimental results

Research questions

• RQ1 Can a stochastic quasi-Newton method be designed to converge globally to stationary points in nonconvex stochastic optimization with only SFO access?
• RQ2 What is the worst-case SFO-calls complexity of such a method to achieve an ε-approximate stationary point?
• RQ3 How does the damped L-BFGS update preserve positive definiteness of the Hessian approximation without requiring convexity?
• RQ4 Does integrating variance reduction (SVRG) into the SdLBFGS framework improve convergence speed and stability?
• RQ5 How does the performance of SdLBFGS-VR compare to SVRG and standard SdLBFGS on real-world nonconvex classification problems?

Key findings

• The proposed SdLBFGS method achieves almost sure convergence to stationary points under appropriate step size conditions.
• The worst-case SFO-calls complexity for achieving $ ext{E}[ orm{\nabla f(x)}^2] \leq \epsilon $ is $ O(\epsilon^{-2}) $, matching the best-known complexity for stochastic gradient methods.
• SdLBFGS-VR, the variance-reduced variant, consistently outperforms both SVRG and standard SdLBFGS on the RCV1 and MNIST datasets.
• Numerical results show that SdLBFGS-VR is relatively insensitive to batch size and memory size, with performance improving slightly with larger memory.
• For the same step size, SdLBFGS-VR achieves better objective value reduction than SVRG, and is more stable than SdLBFGS with constant step size.
• The number of iterations where $ s_{k-1}^T y_{k-1} < 0 $ is significantly reduced when using larger memory sizes or batch sizes, indicating improved curvature estimation.