[Paper Review] A Theoretical and Empirical Comparison of Gradient Approximations in Derivative-Free Optimization

Albert S. Berahas, Liyuan Cao+2 more|arXiv (Cornell University)|May 3, 2019

Advanced Optimization Algorithms Research4 citations

TL;DR

This paper provides a theoretical and empirical comparison of gradient approximation methods—finite differences, linear interpolation, Gaussian smoothing, and spherical smoothing—in derivative-free optimization under bounded noise. It establishes sampling bounds ensuring convergence for line search and fixed-step descent, showing finite difference and interpolation methods outperform smoothing techniques in efficiency and accuracy.

ABSTRACT

In this paper, we analyze several methods for approximating gradients of noisy functions using only function values. These methods include finite differences, linear interpolation, Gaussian smoothing and smoothing on a sphere. The methods differ in the number of functions sampled, the choice of the sample points, and the way in which the gradient approximations are derived. For each method, we derive bounds on the number of samples and the sampling radius which guarantee favorable convergence properties for a line search or fixed step size descent method. To this end, we use the results in [Berahas et al., 2019] and show how each method can satisfy the sufficient conditions, possibly only with some sufficiently large probability at each iteration, as happens to be the case with Gaussian smoothing and smoothing on a sphere. Finally, we present numerical results evaluating the quality of the gradient approximations as well as their performance in conjunction with a line search derivative-free optimization algorithm.

Motivation & Objective

• To analyze and compare the theoretical and empirical performance of gradient approximation methods in derivative-free optimization with bounded noise.
• To derive sampling bounds and radius requirements that guarantee convergence for line search and fixed-step descent methods.
• To evaluate the practical efficiency and accuracy of finite differences, linear interpolation, Gaussian smoothing, and spherical smoothing in noisy optimization.
• To investigate the impact of sampling strategy and step size selection on convergence and performance in derivative-free optimization.
• To provide theoretical justification for using gradient approximations in conjunction with L-BFGS or line search methods under bounded noise assumptions.

Proposed method

• Uses finite differences (forward and central) to estimate gradients via function evaluations at perturbed points along coordinate directions.
• Applies linear interpolation using n linearly independent directions from the current point, with the gradient estimated via a matrix inversion involving the direction vectors.
• Employs Gaussian smoothing by averaging directional derivatives along independent, zero-mean Gaussian directions, with variance tuned for accuracy.
• Implements spherical smoothing by sampling unit vectors uniformly on the n-dimensional sphere and averaging directional derivatives along these directions.
• Derives theoretical bounds on the number of samples and sampling radius to ensure gradient approximations satisfy sufficient descent conditions with high probability.
• Employs line search and L-BFGS methods to evaluate performance, using adaptive and fixed step sizes in numerical experiments.

Experimental results

Research questions

• RQ1 How do different gradient approximation methods perform in terms of theoretical convergence guarantees under bounded noise?
• RQ2 What is the minimum number of function evaluations and sampling radius required for each method to ensure convergence in derivative-free optimization?
• RQ3 How do finite differences, linear interpolation, and smoothing-based methods compare in terms of accuracy and computational efficiency?
• RQ4 What role does the choice of sampling distribution (Gaussian vs. spherical) play in gradient approximation quality and convergence?
• RQ5 How does adaptive step size selection compare to fixed step size in enhancing performance across different gradient approximation methods?

Key findings

• Finite difference and linear interpolation methods require fewer function evaluations and achieve better convergence rates compared to smoothing-based methods.
• Gaussian smoothing and spherical smoothing require significantly more samples (e.g., N = 4n for BSG) to achieve comparable accuracy, but can still guarantee convergence with high probability.
• The best-performing variants used adaptive step size procedures, with LBFGS search directions outperforming steepest descent in most cases.
• Interpolation with random, orthonormal directions outperformed deterministic finite differences in reinforcement learning tasks, likely due to better escape from local minima.
• Line search improved performance over manually tuned fixed step sizes, especially for interpolation and smoothing methods.
• Theoretical bounds on sampling radius and number of samples were derived for each method, ensuring sufficient descent conditions under bounded noise.