[Paper Review] A Tutorial on Bayesian Optimization

Peter I. Frazier|arXiv (Cornell University)|Jul 8, 2018

Advanced Multi-Objective Optimization Algorithms78 references372 citations

TL;DR

This tutorial explains Bayesian optimization (BayesOpt) for expensive, black-box, derivative-free global optimization using Gaussian process regression and acquisition functions. It covers standard and exotic extensions, including parallel evaluations and multi-fidelity settings.

ABSTRACT

Bayesian optimization is an approach to optimizing objective functions that take a long time (minutes or hours) to evaluate. It is best-suited for optimization over continuous domains of less than 20 dimensions, and tolerates stochastic noise in function evaluations. It builds a surrogate for the objective and quantifies the uncertainty in that surrogate using a Bayesian machine learning technique, Gaussian process regression, and then uses an acquisition function defined from this surrogate to decide where to sample. In this tutorial, we describe how Bayesian optimization works, including Gaussian process regression and three common acquisition functions: expected improvement, entropy search, and knowledge gradient. We then discuss more advanced techniques, including running multiple function evaluations in parallel, multi-fidelity and multi-information source optimization, expensive-to-evaluate constraints, random environmental conditions, multi-task Bayesian optimization, and the inclusion of derivative information. We conclude with a discussion of Bayesian optimization software and future research directions in the field. Within our tutorial material we provide a generalization of expected improvement to noisy evaluations, beyond the noise-free setting where it is more commonly applied. This generalization is justified by a formal decision-theoretic argument, standing in contrast to previous ad hoc modifications.

Motivation & Objective

• Introduce Bayesian optimization as a framework for optimizing expensive, derivative-free black-box functions.
• Detail Gaussian process regression as the surrogate model and how to update posteriors with new observations.
• Present acquisition functions (expected improvement, knowledge gradient, entropy-based approaches) and how they guide sampling.
• Discuss exotic extensions (parallel evaluations, multi-fidelity, constraints, multi-task, derivatives) and practical software considerations.

Proposed method

• Model the objective with a Gaussian process to obtain posterior means and variances.
• Define acquisition functions from the GP posterior to select next evaluation points (e.g., EI, KG).
• Derive and implement EI with a closed-form expression involving the posterior mean and variance (Eq. 8).
• Describe alternative acquisition functions: knowledge gradient, entropy search, predictive entropy search.
• Extend BayesOpt to noisy evaluations, parallel evaluations, constraints, multi-fidelity, and other exotic settings.
• Provide guidance on hyperparameter selection for kernels and priors, including MLE, MAP, and fully Bayesian approaches.

Experimental results

Research questions

• RQ1 How can Bayesian optimization efficiently optimize expensive black-box functions using probabilistic surrogates?
• RQ2 What acquisition functions best balance exploration and exploitation in standard and exotic BayesOpt settings?
• RQ3 How can Bayesian optimization be extended to handle noise, parallelism, and multiple information sources?
• RQ4 What are practical considerations for GP hyperparameters and computational strategies in BayesOpt?
• RQ5 How do different acquisition strategies compare in enabling global optimization with limited evaluations?

Key findings

• EI provides a tractable and effective acquisition framework with a closed-form expression.
• KG offers a decision-theoretic alternative that accounts for potential value of information from future observations.
• Alternative acquisitions like entropy search and predictive entropy search address exotic settings and constraints.
• Extensions to parallel evaluations, multi-fidelity, and derivative information broaden BayesOpt’s applicability to real-world problems.
• A generalization of EI to noisy evaluations is proposed and justified via a formal decision-theoretic argument.