[Paper Review] Sparse Stochastic Zeroth-Order Optimization with an Application to Bandit Structured Prediction

Artem Sokolov, Julian Hitschler+2 more|arXiv (Cornell University)|Jun 12, 2018

Advanced Bandit Algorithms ResearchDecision Sciences38 references4 citations

TL;DR

This paper proposes sparse stochastic zeroth-order (SZO) optimization to reduce the dimensionality bottleneck in gradient-free learning, leveraging sparsity in structured prediction features. By perturbing only active features, the method achieves convergence rates close to stochastic first-order methods, with empirical results showing near-SFO performance on noun-phrase chunking and superior results on machine translation using non-differentiable MAP criteria.

ABSTRACT

Stochastic zeroth-order (SZO), or gradient-free, optimization allows to optimize arbitrary functions by relying only on function evaluations under parameter perturbations, however, the iteration complexity of SZO methods suffers a factor proportional to the dimensionality of the perturbed function. We show that in scenarios with natural sparsity patterns as in structured prediction applications, this factor can be reduced to the expected number of active features over input-output pairs. We give a general proof that applies sparse SZO optimization to Lipschitz-continuous, nonconvex, stochastic objectives, and present an experimental evaluation on linear bandit structured prediction tasks with sparse word-based feature representations that confirm our theoretical results.

Motivation & Objective

• To address the high iteration complexity of stochastic zeroth-order (SZO) optimization, which scales with full parameter dimensionality.
• To show that in sparse structured prediction settings, the iteration complexity can be reduced to the expected number of active features rather than full dimensionality.
• To develop and evaluate SZO algorithms that use sparse perturbations for improved efficiency in bandit structured prediction.
• To investigate the feasibility of applying non-differentiable loss criteria (e.g., MAP) at both training and test time using SZO, which is infeasible with standard SFO methods.

Proposed method

• Theoretical analysis extends prior SZO work to non-convex, Lipschitz-continuous, stochastic objectives using sparse parameter perturbations.
• Three SZO algorithms are proposed: two-point function evaluation, baseline comparison, and function comparison, each designed to reduce feedback complexity.
• Sparse perturbations are applied only to active features in input-output pairs, such as n-grams in sequence labeling tasks.
• The method relies on Gaussian smoothing to ensure Lipschitz continuity of the smoothed objective, enabling gradient approximation via function value comparisons.
• Hyperparameters (learning rate h and exploration parameter μ) are tuned on development sets, with results averaged over three random seeds.
• The approach is evaluated on noun-phrase chunking (F1 score) and statistical machine translation (BLEU score) using both dense and sparse perturbations.

Experimental results

Research questions

• RQ1 Can the dimensionality bottleneck in SZO optimization be mitigated by exploiting sparsity in structured prediction feature spaces?
• RQ2 Does sparse parameter perturbation lead to convergence rates comparable to stochastic first-order methods in practice?
• RQ3 Can non-differentiable loss functions like MAP be used consistently at training and test time in SZO, enabling improved performance over SFO methods?
• RQ4 How does feedback complexity (e.g., one-point vs. two-point function evaluations) affect convergence speed and final performance in SZO for structured prediction?

Key findings

• The SPARSE two-point update rule achieved a test F1 score of 0.888 on noun-phrase chunking, approaching the SFO baseline of 0.908.
• The SPARSE baseline comparison update achieved the highest test F1 score of 0.869 on chunking, outperforming the ALL-perturbation baseline comparison (0.819).
• On machine translation, the best SZO result (two-point updates with sparse perturbations) achieved a BLEU score of 0.273, outperforming the SFO result of 0.263 by 0.01 BLEU points.
• The SZO method with sparse perturbations and two-point feedback reached optimal performance at 100k iterations, significantly faster than the SFO method.
• Using the non-differentiable MAP criterion at both training and test time with SZO led to a 1.6 BLEU point improvement over the out-of-domain baseline, demonstrating a unique advantage over SFO.
• All SZO methods with sparse perturbations showed improved convergence speed compared to dense perturbations, confirming the theoretical reduction in effective dimensionality.