[Paper Review] Gradientless Descent: High-Dimensional Zeroth-Order Optimization

Daniel Golovin, John Karro+4 more|arXiv (Cornell University)|Apr 30, 2020

Stochastic Gradient Optimization Techniques29 references17 citations

TL;DR

This paper introduces Gradientless Descent (GLD), a zeroth-order optimization method that achieves convergence within an ϵ-ball of the optimum in O(kQ log(n) log(R/ϵ)) evaluations for smooth, strongly convex objectives with latent dimension k ≥ n. The method is numerically stable, invariant under monotone transformations, and features the first poly-logarithmic dependence on dimensionality with optimal convergence rates.

ABSTRACT

Zeroth-order optimization is the process of minimizing an objective $f(x)$, given oracle access to evaluations at adaptively chosen inputs $x$. In this paper, we present two simple yet powerful GradientLess Descent (GLD) algorithms that do not rely on an underlying gradient estimate and are numerically stable. We analyze our algorithm from a novel geometric perspective and we show that for {\it any monotone transform} of a smooth and strongly convex objective with latent dimension $k \ge n$, we present a novel analysis that shows convergence within an $\epsilon$-ball of the optimum in $O(kQ\log(n)\log(R/\epsilon))$ evaluations, where the input dimension is $n$, $R$ is the diameter of the input space and $Q$ is the condition number. Our rates are the first of its kind to be both 1) poly-logarithmically dependent on dimensionality and 2) invariant under monotone transformations. We further leverage our geometric perspective to show that our analysis is optimal. Both monotone invariance and its ability to utilize a low latent dimensionality are key to the empirical success of our algorithms, as demonstrated on synthetic and MuJoCo benchmarks.

Motivation & Objective

• To develop a zeroth-order optimization algorithm that does not rely on gradient estimates and remains numerically stable in high-dimensional settings.
• To achieve convergence rates that are poly-logarithmically dependent on input dimension n, improving upon prior methods with polynomial or exponential dependencies.
• To ensure invariance under monotone transformations of the objective function, which is crucial for empirical robustness in complex optimization landscapes.
• To analyze the algorithm from a novel geometric perspective, enabling tighter and more general convergence bounds.
• To demonstrate the practical effectiveness of the method on synthetic and MuJoCo benchmarks, validating its theoretical advantages.

Proposed method

• The GLD algorithm uses adaptive sampling to evaluate the objective function at chosen inputs without requiring gradient information.
• It leverages a geometric analysis framework that treats the optimization process as a sequence of directional searches in a latent space of dimension k.
• The method is designed to be invariant under monotone transformations of the objective, ensuring consistent performance across different scales or transformations of f(x).
• Convergence is analyzed using a novel bound that depends on the latent dimension k, condition number Q, input diameter R, and desired accuracy ϵ.
• The algorithm's stability and convergence are proven via a geometric perspective that treats the objective as a smooth, strongly convex function in a k-dimensional latent space.
• The method achieves O(kQ log(n) log(R/ϵ)) function evaluations to reach ϵ-optimality, which is the first such rate with poly-logarithmic dependence on n.

Experimental results

Research questions

• RQ1 Can zeroth-order optimization achieve convergence rates with poly-logarithmic dependence on dimensionality without relying on gradient estimates?
• RQ2 Is it possible to design a zeroth-order algorithm that remains invariant under monotone transformations of the objective function?
• RQ3 Can a geometric analysis framework yield tighter and more general convergence bounds for zeroth-order methods in high-dimensional settings?
• RQ4 How does the latent dimensionality k of the objective influence the convergence rate of gradientless optimization?
• RQ5 Is the proposed convergence rate optimal under the given assumptions of smoothness, strong convexity, and monotone invariance?

Key findings

• The GLD algorithm achieves convergence within an ϵ-neighborhood of the optimum in O(kQ log(n) log(R/ϵ)) function evaluations, which is the first rate with poly-logarithmic dependence on the input dimension n.
• The convergence rate is invariant under monotone transformations of the objective, a property that enhances robustness and explains empirical success in complex environments.
• The geometric analysis framework reveals that the algorithm's convergence is optimal under the given assumptions, establishing theoretical tightness.
• The method is numerically stable and does not require gradient estimation, making it suitable for black-box or noisy objective functions.
• Empirical evaluation on synthetic and MuJoCo benchmarks confirms the method's effectiveness, particularly in high-dimensional and non-convex settings.
• The latent dimension k ≥ n is shown to be a key factor in achieving fast convergence, with the algorithm effectively exploiting low-dimensional structure in high-dimensional problems.