[Paper Review] Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions

Amar Shah, Zoubin Ghahramani|arXiv (Cornell University)|Nov 23, 2015

Gaussian Processes and Bayesian InferenceComputer Science32 references66 citations

TL;DR

This paper introduces Parallel Predictive Entropy Search (PPES), a non-greedy Bayesian optimization algorithm for selecting batches of points to evaluate in parallel, aiming to maximize information gain about the global maximizer of expensive black-box functions. PPES outperforms greedy batch methods on synthetic and real-world problems in robotics, aerospace, and machine learning, demonstrating superior exploration and faster convergence due to its information-theoretic, non-greedy batch selection strategy.

ABSTRACT

We develop parallel predictive entropy search (PPES), a novel algorithm for Bayesian optimization of expensive black-box objective functions. At each iteration, PPES aims to select a batch of points which will maximize the information gain about the global maximizer of the objective. Well known strategies exist for suggesting a single evaluation point based on previous observations, while far fewer are known for selecting batches of points to evaluate in parallel. The few batch selection schemes that have been studied all resort to greedy methods to compute an optimal batch. To the best of our knowledge, PPES is the first non-greedy batch Bayesian optimization strategy. We demonstrate the benefit of this approach in optimization performance on both synthetic and real world applications, including problems in machine learning, rocket science and robotics.

Motivation & Objective

• To address the challenge of efficiently optimizing expensive black-box objective functions when multiple evaluations can be performed in parallel.
• To overcome the limitations of greedy batch selection strategies, which often under-explore the input space and fail to capture global structure.
• To develop a non-greedy, information-theoretic approach that selects batches of points to maximize expected reduction in uncertainty about the location of the global maximizer.
• To evaluate the method on real-world problems where function evaluations are costly and noisy, including robotics, rocket science, and machine learning hyperparameter tuning.

Proposed method

• PPES selects a batch of Q points by maximizing the expected reduction in entropy (information gain) about the location of the global maximizer, rather than maximizing immediate improvement.
• It extends predictive entropy search to batch settings by modeling the joint uncertainty over the location of the maximizer and the function values at candidate points.
• The method uses Monte Carlo sampling to approximate the intractable expectation of information gain over the joint distribution of the maximizer and function values.
• It formulates the acquisition function as the expected reduction in entropy of the posterior distribution over the maximizer location after observing a batch of points.
• The algorithm optimizes the batch by solving a high-dimensional optimization problem over the Q-point input space, using gradient-based methods from multiple random initializations.
• Unlike greedy methods, PPES does not sequentially select points; instead, it jointly optimizes the entire batch to maximize global information gain.

Experimental results

Research questions

• RQ1 Can a non-greedy batch Bayesian optimization strategy outperform greedy methods in terms of convergence speed and global exploration?
• RQ2 Does maximizing information gain about the location of the global maximizer lead to better optimization performance than maximizing expected improvement?
• RQ3 How does PPES perform on multi-modal, discontinuous, and noisy objective functions where greedy methods often fail?
• RQ4 What is the impact of batch size on optimization performance when using a non-greedy, information-theoretic acquisition function?

Key findings

• PPES consistently outperforms greedy batch methods such as SM-UCB, GP-BUCB, and EI-MCMC in terms of median immediate regret across all benchmark functions.
• On the hydrogen function, PPES shows a rapid drop in regret after initial batches, indicating superior long-term exploration due to non-greedy selection.
• For the rocket function, which features a strong discontinuity and zero returns for infeasible configurations, PPES achieved fewer zero-function-value evaluations than competing methods.
• PPES made better recommendations for the global maximizer (i.e., better estimates of x*) than other methods, even when its individual evaluations had lower function values, due to its focus on information gain.
• The performance gain of PPES was most pronounced on complex, multi-modal functions, where greedy methods suffered from premature convergence and poor exploration.
• The relative improvement from increasing batch size from Q=2 to Q=4 was smaller for the robot function, likely due to suboptimal optimization of the acquisition function in high-dimensional batch space.