[Paper Review] Parallel Bayesian Global Optimization of Expensive Functions

Jialei Wang, Scott Clark+2 more|arXiv (Cornell University)|Feb 16, 2016

Advanced Multi-Objective Optimization Algorithms31 references67 citations

TL;DR

This paper proposes a stochastic gradient-based method for parallel Bayesian global optimization of expensive functions, using infinitesimal perturbation analysis (IPA) to construct an unbiased gradient estimator for the multi-points expected improvement (q-EI). The approach enables efficient, scalable optimization across multiple evaluations and outperforms closed-form methods in computation time for q ≥ 4, especially with GPU acceleration.

ABSTRACT

We consider parallel global optimization of derivative-free expensive-to-evaluate functions, and propose an efficient method based on stochastic approximation for implementing a conceptual Bayesian optimization algorithm proposed by Ginsbourger et al. (2007). At the heart of this algorithm is maximizing the information criterion called the "multi-points expected improvement'', or the q-EI. To accomplish this, we use infinitessimal perturbation analysis (IPA) to construct a stochastic gradient estimator and show that this estimator is unbiased. We also show that the stochastic gradient ascent algorithm using the constructed gradient estimator converges to a stationary point of the q-EI surface, and therefore, as the number of multiple starts of the gradient ascent algorithm and the number of steps for each start grow large, the one-step Bayes optimal set of points is recovered. We show in numerical experiments that our method for maximizing the q-EI is faster than methods based on closed-form evaluation using high-dimensional integration, when considering many parallel function evaluations, and is comparable in speed when considering few. We also show that the resulting one-step Bayes optimal algorithm for parallel global optimization finds high-quality solutions with fewer evaluations than a heuristic based on approximately maximizing the q-EI. A high-quality open source implementation of this algorithm is available in the open source Metrics Optimization Engine (MOE).

Motivation & Objective

• Address the challenge of efficiently maximizing the q-EI criterion in parallel Bayesian optimization, which is computationally expensive due to high-dimensional integration.
• Overcome the limitations of existing closed-form and heuristic methods for q-EI maximization, which scale poorly with the number of parallel evaluations (q).
• Develop a scalable, differentiable, and parallelizable method for q-EI maximization suitable for high-dimensional and large-q optimization settings.
• Enable one-step Bayes-optimal selection of multiple points in expensive global optimization by accurately approximating the q-EI gradient.
• Provide a high-performance, open-source implementation in the Metrics Optimization Engine (MOE) for practical deployment in real-world applications.

Proposed method

• Use infinitesimal perturbation analysis (IPA) to construct an unbiased stochastic gradient estimator for the q-EI objective function.
• Implement a stochastic gradient ascent algorithm that leverages the IPA-based gradient estimator to optimize the q-EI surface.
• Utilize Monte Carlo sampling with a large number of replications to reduce gradient variance and ensure high-accuracy estimation.
• Enable GPU acceleration by exploiting the trivial parallelism inherent in Monte Carlo gradient estimation, significantly improving scalability with q.
• Apply a decreasing stepsize sequence in stochastic gradient ascent to automatically balance exploration and exploitation, focusing computational effort near convergence.
• Integrate the method into the open-source Metrics Optimization Engine (MOE), supporting both CPU and GPU deployment for production use.

Experimental results

Research questions

• RQ1 Can a stochastic gradient-based approach with IPA provide an unbiased and scalable alternative to closed-form q-EI gradient computation?
• RQ2 How does the computational complexity of the proposed stochastic gradient estimator scale with the number of parallel evaluations (q) compared to existing closed-form methods?
• RQ3 Does the stochastic gradient ascent algorithm with the IPA estimator converge to a stationary point of the q-EI surface, ensuring theoretical consistency?
• RQ4 Can the method achieve faster computation times than closed-form evaluation for q ≥ 4, particularly when leveraging GPU acceleration?
• RQ5 How does the performance of the proposed method compare to heuristic q-EI maximization strategies in terms of solution quality and function evaluations?

Key findings

• The IPA-based stochastic gradient estimator is unbiased and enables convergence of the stochastic gradient ascent algorithm to a stationary point of the q-EI surface.
• The MOE-qEI implementation with GPU acceleration computes gradients faster than the closed-form method (Benchmark 2) for q ≥ 4, with significantly better scaling as q increases.
• Even with only 10,000 Monte Carlo samples, the gradient variance remains below 10⁻⁷, ensuring high accuracy at a fraction of the computational cost.
• The CPU-only version of the stochastic gradient estimator runs faster than the GPU-based closed-form method when q ≥ 4, demonstrating strong performance without specialized hardware.
• The method achieves higher-quality optimization solutions with fewer function evaluations than heuristic q-EI maximization approaches.
• The stochastic gradient approach enables adaptive computation, reducing gradient estimation effort early in optimization and focusing on accuracy near convergence.