[Paper Review] The Parallel Knowledge Gradient Method for Batch Bayesian Optimization

Jian Wu, Peter I. Frazier|arXiv (Cornell University)|Jan 1, 2016

Machine Learning and AlgorithmsComputer Science86 citations

TL;DR

This paper introduces the Parallel Knowledge Gradient (PKG) method, a batch Bayesian optimization algorithm that selects the one-step Bayes-optimal batch of points for parallel evaluation. By efficiently computing this optimal batch, PKG accelerates convergence to global optima—especially under noisy evaluations—outperforming prior batch Bayesian optimization methods on synthetic functions and machine learning hyperparameter tuning.

ABSTRACT

In many applications of black-box optimization, one can evaluate multiple points simultaneously, e.g. when evaluating the performances of several different neural network architectures in a parallel computing environment. In this paper, we develop a novel batch Bayesian optimization algorithm --- the parallel knowledge gradient method. By construction, this method provides the one-step Bayes optimal batch of points to sample. We provide an efficient strategy for computing this Bayes-optimal batch of points, and we demonstrate that the parallel knowledge gradient method finds global optima significantly faster than previous batch Bayesian optimization algorithms on both synthetic test functions and when tuning hyperparameters of practical machine learning algorithms, especially when function evaluations are noisy.

Motivation & Objective

• To address the challenge of efficiently selecting batches of points for parallel evaluation in black-box optimization, especially in settings with noisy function evaluations.
• To develop a method that identifies the one-step Bayes-optimal batch of points, maximizing expected improvement in the objective function.
• To enable faster convergence to global optima in hyperparameter tuning and other applications requiring parallel function evaluations.
• To provide an efficient computational strategy for identifying the optimal batch, overcoming the high computational cost of prior batch Bayesian optimization approaches.

Proposed method

• The method formulates the batch selection problem as a knowledge gradient optimization, where the expected improvement in the posterior mean of the objective function is maximized over candidate batches.
• It derives a closed-form approximation for the knowledge gradient in the batch setting, enabling efficient computation of the optimal batch without exhaustive enumeration.
• The algorithm uses a greedy selection strategy based on the knowledge gradient to iteratively build the batch, ensuring computational tractability for large batches.
• It incorporates uncertainty estimates from a Gaussian process surrogate model to balance exploration and exploitation in the batch selection process.
• The method is designed to be scalable and efficient, allowing real-time batch selection in parallel computing environments.
• The approach is grounded in Bayesian decision theory, ensuring that the selected batch is optimal in the one-step Bayes sense.

Experimental results

Research questions

• RQ1 How can we efficiently compute the one-step Bayes-optimal batch of points for parallel function evaluation in Bayesian optimization?
• RQ2 Can a knowledge gradient-based approach outperform existing batch Bayesian optimization methods in terms of convergence speed and global optimality?
• RQ3 How does the proposed method perform under noisy function evaluations compared to prior batch Bayesian optimization algorithms?
• RQ4 What is the scalability of the method in terms of batch size and dimensionality of the search space?

Key findings

• The Parallel Knowledge Gradient method achieves significantly faster convergence to global optima compared to previous batch Bayesian optimization algorithms on synthetic test functions.
• On hyperparameter tuning tasks for practical machine learning models, PKG reduces the number of function evaluations required to reach optimal performance.
• The method demonstrates superior performance in noisy evaluation settings, maintaining robustness and efficiency where other methods degrade.
• The computational strategy for batch selection enables efficient scaling to larger batches, making it suitable for high-performance computing environments.
• Empirical results show that PKG consistently finds better solutions with fewer evaluations than baseline methods, especially in high-noise regimes.