[Paper Review] A gradient estimator via L1-randomization for online zero-order optimization with two point feedback

Arya Akhavan, Evgenii Chzhen+2 more|arXiv (Cornell University)|May 27, 2022

Sparse and Compressive Sensing Techniques4 citations

TL;DR

This paper introduces a novel L1-randomization-based gradient estimator for online zero-order optimization with two-point feedback, enabling improved regret bounds in convex, Lipschitz settings. The method achieves adaptive, data-driven performance that matches or exceeds state-of-the-art results—particularly improving the rate by a √log(d) factor under ℓ₁-norm Lipschitz assumptions and canceling noise.

ABSTRACT

This work studies online zero-order optimization of convex and Lipschitz functions. We present a novel gradient estimator based on two function evaluations and randomization on the $\ell_1$-sphere. Considering different geometries of feasible sets and Lipschitz assumptions we analyse online dual averaging algorithm with our estimator in place of the usual gradient. We consider two types of assumptions on the noise of the zero-order oracle: canceling noise and adversarial noise. We provide an anytime and completely data-driven algorithm, which is adaptive to all parameters of the problem. In the case of canceling noise that was previously studied in the literature, our guarantees are either comparable or better than state-of-the-art bounds obtained by Duchi et al. (2015) and Shamir (2017) for non-adaptive algorithms. Our analysis is based on deriving a new weighted Poincaré type inequality for the uniform measure on the $\ell_1$-sphere with explicit constants, which may be of independent interest.

Motivation & Objective

• Address the challenge of online zero-order optimization when only function values (with noise) are accessible, not gradients.
• Overcome limitations of prior methods relying on ℓ₂ or ℓ∞-randomization by introducing an ℓ₁-sphere-based gradient estimator.
• Develop an anytime, data-driven algorithm adaptive to unknown problem parameters such as dimension and Lipschitz constants.
• Analyze performance under both canceling noise (where noise terms cancel) and adversarial noise (worst-case noise).
• Establish theoretical guarantees that improve upon existing bounds, especially in ℓ₁-norm settings, and derive a new Poincaré-type inequality for the ℓ₁-sphere.

Proposed method

• Propose a two-point feedback gradient estimator using random vectors uniformly distributed on the ℓ₁-sphere, enabling unbiased estimation of subgradients.
• Integrate the estimator into the online dual averaging (ODA) framework to minimize cumulative regret in convex, Lipschitz settings.
• Use a weighted Poincaré-type inequality for the uniform measure on the ℓ₁-sphere to control variance and derive sharp concentration bounds.
• Design an adaptive algorithm that tunes step-sizes based on observed function values, eliminating the need for prior knowledge of problem parameters.
• Employ a novel parameterization strategy that dynamically adjusts to the observed noise and function variation, ensuring anytime performance.
• Analyze both canceling and adversarial noise models, showing that the algorithm maintains strong regret guarantees under both.

Experimental results

Research questions

• RQ1 Can a gradient estimator based on ℓ₁-randomization outperform existing ℓ₂- or ℓ∞-based estimators in online zero-order optimization?
• RQ2 What is the optimal regret rate achievable with two-point feedback under ℓ₁-norm Lipschitz assumptions and canceling noise?
• RQ3 Can an adaptive, data-driven algorithm be constructed that performs well without prior knowledge of dimension, Lipschitz constant, or noise level?
• RQ4 Is the proposed estimator robust under adversarial noise, and what is the resulting regret rate compared to the canceling noise case?
• RQ5 Can a new Poincaré-type inequality for the uniform measure on the ℓ₁-sphere be derived with explicit constants, and is it useful for analyzing zero-order optimization?

Key findings

• The proposed L1-randomization gradient estimator achieves a regret bound that improves by a √log(d) factor over prior work when the objective function is ℓ₁-Lipschitz and the feasible set is the probability simplex.
• For ℓ₂-Lipschitz functions, the regret bound matches the state-of-the-art result of Shamir (2017) up to absolute constants, with explicit control over the dependence on dimension.
• Under canceling noise, the algorithm achieves a minimax-optimal regret rate of O(√(T log d)) for q = 2 and O(√(T log d)) up to a log(d) factor for q = 1, matching known lower bounds.
• The adaptive version of the algorithm achieves performance comparable to the non-adaptive one under canceling noise and only slightly worse under adversarial noise.
• The paper derives a new weighted Poincaré inequality for the uniform measure on the ℓ₁-sphere with explicit constants, which is a key technical contribution with potential broader applications.
• Under adversarial noise, the regret rate is O(T^{3/4}), which is the first such guarantee for an adaptive, data-driven algorithm in this setting, though optimality remains open.