[Paper Review] Zeroth-order Asynchronous Doubly Stochastic Algorithm with Variance Reduction

Bin Gu, Zhouyuan Huo+1 more|arXiv (Cornell University)|Dec 5, 2016

Advanced Bandit Algorithms Research18 references11 citations

TL;DR

This paper proposes AsyDSZOVR, an asynchronous doubly stochastic zeroth-order optimization algorithm that integrates variance reduction to achieve a convergence rate of $O(1/T)$, significantly improving upon the $O(1/√{T})$ rate of existing asynchronous and sequential stochastic zeroth-order methods. The method enables efficient large-scale optimization in derivative-free settings by simultaneously sampling mini-batches of data and coordinate subsets in a lock-free parallel environment.

ABSTRACT

Zeroth-order (derivative-free) optimization attracts a lot of attention in machine learning, because explicit gradient calculations may be computationally expensive or infeasible. To handle large scale problems both in volume and dimension, recently asynchronous doubly stochastic zeroth-order algorithms were proposed. The convergence rate of existing asynchronous doubly stochastic zeroth order algorithms is $O(\frac{1}{\sqrt{T}})$ (also for the sequential stochastic zeroth-order optimization algorithms). In this paper, we focus on the finite sums of smooth but not necessarily convex functions, and propose an asynchronous doubly stochastic zeroth-order optimization algorithm using the accelerated technology of variance reduction (AsyDSZOVR). Rigorous theoretical analysis show that the convergence rate can be improved from $O(\frac{1}{\sqrt{T}})$ the best result of existing algorithms to $O(\frac{1}{T})$. Also our theoretical results is an improvement to the ones of the sequential stochastic zeroth-order optimization algorithms.

Motivation & Objective

• Address the lack of fast-converging asynchronous zeroth-order optimization methods for large-scale, derivative-free machine learning problems.
• Overcome the slow convergence of existing $O(1/\sqrt{T})$-rate stochastic zeroth-order algorithms in both sequential and asynchronous settings.
• Develop a scalable, lock-free parallel algorithm that handles high-dimensional and high-volume data without requiring gradient information.
• Achieve accelerated convergence via variance reduction techniques in an asynchronous, doubly stochastic framework.

Proposed method

• Proposes AsyDSZOVR, an asynchronous, doubly stochastic zeroth-order algorithm that randomly samples mini-batches of data and coordinate subsets in parallel.
• Employs a variance-reduced gradient estimator using a control variate mechanism to reduce noise in gradient approximations.
• Uses a recursive update rule with adaptive step sizes $\gamma = \frac{u_0 b}{\widetilde{L} l^\alpha}$ and momentum-like terms to accelerate convergence.
• Introduces a Lyapunov function-based analysis to bound the expected norm of the gradient, ensuring convergence under asynchronous updates.
• Derives a lower bound $\widetilde{\Gamma}$ on the progress per iteration, which is used to establish the $O(1/T)$ convergence rate.
• Applies a lock-free parallel computing model where threads independently read, compute, and update shared parameters without synchronization.

Experimental results

Research questions

• RQ1 Can variance reduction techniques be effectively adapted to asynchronous, zeroth-order optimization to achieve faster convergence?
• RQ2 What is the theoretical convergence rate of an asynchronous, doubly stochastic zeroth-order algorithm with variance reduction?
• RQ3 Can the convergence rate be improved beyond $O(1/\sqrt{T})$ in both sequential and asynchronous settings?
• RQ4 How does the proposed method handle large-scale problems in terms of data volume and dimensionality?
• RQ5 What conditions ensure convergence in a lock-free, asynchronous parallel environment without explicit gradient computation?

Key findings

• The proposed AsyDSZOVR algorithm achieves a convergence rate of $O(1/T)$, improving upon the $O(1/\sqrt{T})$ rate of existing asynchronous and sequential stochastic zeroth-order algorithms.
• Theoretical analysis proves that the expected gradient norm decays as $O(1/T)$, with the bound $\frac{1}{T}\sum_{s=0}^{S-1}\sum_{t=0}^{m-1}\mathbb{E}\left\|\nabla f(x^{s+1}_t)\right\|^2 \leq \frac{\widetilde{L}l^\alpha(f(x^0) - \mathbb{E}(f(x^S)))}{\sigma b T}$ when $\omega = 0$.
• The convergence rate improvement is achieved through variance reduction and adaptive step size scheduling, even in the absence of gradient information.
• The algorithm is scalable and efficient in shared-memory, lock-free environments, enabling parallel updates without synchronization overhead.
• The analysis establishes a lower bound $\widetilde{\Gamma} \geq \frac{\sigma b}{\widetilde{L}l^\alpha}$, which ensures the $O(1/T)$ convergence under appropriate parameter choices.
• The method is applicable to finite sums of smooth, possibly non-convex functions, covering common machine learning problems like logistic regression and ridge regression.