[Paper Review] Bayesian Optimization with Gradients

Jian Wu, Matthias Poloczek+2 more|arXiv (Cornell University)|Mar 13, 2017

Advanced Bandit Algorithms Research34 references95 citations

TL;DR

Introduces d-KG, a derivative-enabled knowledge-gradient acquisition for Bayesian optimization that uses gradient information (even noisy or partial) to achieve more efficient global optimization, with a fast discretization-free computation and theoretical guarantees.

ABSTRACT

Bayesian optimization has been successful at global optimization of expensive-to-evaluate multimodal objective functions. However, unlike most optimization methods, Bayesian optimization typically does not use derivative information. In this paper we show how Bayesian optimization can exploit derivative information to decrease the number of objective function evaluations required for good performance. In particular, we develop a novel Bayesian optimization algorithm, the derivative-enabled knowledge-gradient (dKG), for which we show one-step Bayes-optimality, asymptotic consistency, and greater one-step value of information than is possible in the derivative-free setting. Our procedure accommodates noisy and incomplete derivative information, comes in both sequential and batch forms, and can optionally reduce the computational cost of inference through automatically selected retention of a single directional derivative. We also compute the d-KG acquisition function and its gradient using a novel fast discretization-free technique. We show d-KG provides state-of-the-art performance compared to a wide range of optimization procedures with and without gradients, on benchmarks including logistic regression, deep learning, kernel learning, and k-nearest neighbors.

Motivation & Objective

• Leverage gradient information to improve Bayesian optimization efficiency.
• Develop a derivative-enabled knowledge-gradient (d-KG) acquisition that handles noisy/incomplete gradients.
• Provide a fast, discretization-free method to compute and optimize the d-KG acquisition.
• Prove theoretical properties: one-step Bayes-optimality, increased VOI, and asymptotic consistency.

Proposed method

• Model the objective as a Gaussian process with joint function and gradient observations.
• Extend the GP to a multi-output process for (f(x), ∇f(x)) with mean and kernel tilde{μ}, tilde{K}.
• Define d-KG as the expected reduction in the minimum posterior mean after observing a batch of derivatives.
• Allow observation of only function values, gradients in certain directions, or incomplete derivatives.
• Provide a discretization-free, unbiased gradient estimator for d-KG to enable stochastic gradient ascent for outer optimization.
• Incorporate a fully Bayesian treatment of hyperparameters by averaging d-KG over multiple GP hyperparameter samples.

Experimental results

Research questions

• RQ1 How can derivative information (full, partial, or noisy) be incorporated into Bayesian optimization?
• RQ2 Does the derivative-enabled knowledge-gradient (d-KG) provide higher value of information than derivative-free methods?
• RQ3 Can d-KG be computed efficiently in continuous domains without discretization, and still be theoretically sound?
• RQ4 What are the empirical benefits of d-KG across synthetic benchmarks and real ML tasks (kernel learning, logistic regression, deep learning, KNN)?
• RQ5 How does d-KG perform in sequential and batch settings, and with directional derivative selection?

Key findings

• d-KG yields higher one-step value of information than the derivative-free KG under mild conditions.
• The acquisition function can be computed with a fast discretization-free method, enabling scalable optimization.
• d-KG is one-step Bayes-optimal and asymptotically consistent on finite feasible sets.
• Empirical results show state-of-the-art performance for d-KG across synthetic benchmarks and real tasks (kernel learning, logistic regression, deep learning, KNN).
• Using directed derivatives (even noisy or partial) improves performance over gradient-free approaches in multiple benchmarks.