[Paper Review] Scalable Global Optimization via Local Bayesian Optimization

David Eriksson, Michael Pearce+3 more|arXiv (Cornell University)|Oct 3, 2019

Advanced Bandit Algorithms ResearchDecision Sciences144 citations

TL;DR

The paper introduces TuRBO, a local-model based Bayesian optimization framework that uses multiple trust regions with independent local GPs and an implicit bandit to allocate samples globally, improving performance on high-dimensional, expensive black-box functions.

ABSTRACT

Bayesian optimization has recently emerged as a popular method for the sample-efficient optimization of expensive black-box functions. However, the application to high-dimensional problems with several thousand observations remains challenging, and on difficult problems Bayesian optimization is often not competitive with other paradigms. In this paper we take the view that this is due to the implicit homogeneity of the global probabilistic models and an overemphasized exploration that results from global acquisition. This motivates the design of a local probabilistic approach for global optimization of large-scale high-dimensional problems. We propose the $ exttt{TuRBO}$ algorithm that fits a collection of local models and performs a principled global allocation of samples across these models via an implicit bandit approach. A comprehensive evaluation demonstrates that $ exttt{TuRBO}$ outperforms state-of-the-art methods from machine learning and operations research on problems spanning reinforcement learning, robotics, and the natural sciences.

Motivation & Objective

• Motivate the global optimization of high-dimensional, expensive black-box functions and address the limitations of global surrogate models.
• Propose a scalable, local Bayesian optimization framework that handles heterogeneity and high dimensionality without over-exploration.
• Demonstrate empirical superiority of TuRBO across robotics, reinforcement learning, cosmology, and synthetic benchmarks.

Proposed method

• Maintain multiple local Gaussian process surrogates, each operating in its own trust region (TR).
• Dynamically adjust TR size via success/failure counts to balance exploration and exploitation.
• Use Thompson sampling to select batch candidates within and across TRs, enabling implicit multi-armed bandit style global allocation.
• Treat each TR as an independent bandit arm to guide sample allocation toward promising regions.
• Compare against a wide range of baselines including BO variants, CMA-ES, and random search across diverse tasks.

Experimental results

Research questions

• RQ1 Does a collection of local probabilistic models with an implicit bandit allocation outperform global surrogate models on high-dimensional, expensive functions?
• RQ2 Can dynamic trust-region sizing and parallel local searches yield scalable, robust global optimization in practice?
• RQ3 How does TuRBO compare to state-of-the-art Bayesian optimization, evolutionary strategies, and stochastic optimization across real-world tasks?
• RQ4 What is the impact of batch size on wall-clock efficiency and solution quality for TuRBO?
• RQ5 Do local models provide advantages over a single global model in terms of predictive accuracy and hyperparameter learnability?

Key findings

• TuRBO consistently finds excellent solutions and often outperforms baselines across robotics, reinforcement learning, and natural sciences.
• A linear speed-up is observed with increasing batch sizes, without sacrificing solution quality.
• Local GPs offer better predictive performance and more flexible hyperparameterization than a single global GP.
• Multiple small TRs capture multimodality and diverse optima, enabling effective global exploration via bandit-like allocation.
• TuRBO with multiple regions (m>1) generally outperforms single-region variants, especially on high-dimensional problems.
• Large-batch experimentation yields near-linear wall-clock speedups while maintaining solution quality.