[Paper Review] BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization

Maximilian Balandat, Brian Karrer+5 more|arXiv (Cornell University)|Oct 14, 2019

Machine Learning and Data ClassificationComputer Science203 citations

TL;DR

BoTorch introduces a modular MC Bayesian optimization framework built on PyTorch, with a novel sample average approximation optimization, auto-differentiation, variance reduction, and a one-shot KG formulation, achieving improved sample efficiency and scalability over existing libraries.

ABSTRACT

Bayesian optimization provides sample-efficient global optimization for a broad range of applications, including automatic machine learning, engineering, physics, and experimental design. We introduce BoTorch, a modern programming framework for Bayesian optimization that combines Monte-Carlo (MC) acquisition functions, a novel sample average approximation optimization approach, auto-differentiation, and variance reduction techniques. BoTorch's modular design facilitates flexible specification and optimization of probabilistic models written in PyTorch, simplifying implementation of new acquisition functions. Our approach is backed by novel theoretical convergence results and made practical by a distinctive algorithmic foundation that leverages fast predictive distributions, hardware acceleration, and deterministic optimization. We also propose a novel "one-shot" formulation of the Knowledge Gradient, enabled by a combination of our theoretical and software contributions. In experiments, we demonstrate the improved sample efficiency of BoTorch relative to other popular libraries.

Motivation & Objective

• Provide a modular, differentiable framework for Monte-Carlo Bayesian optimization that leverages modern hardware and PyTorch-based models.
• Introduce and analyze sample average approximation (SAA) for optimizing MC acquisition functions, with convergence guarantees.
• Present a one-shot Knowledge Gradient (OKG) look-ahead acquisition and show its practical benefits.
• Demonstrate performance and scalability improvements over existing BO libraries through extensive experiments.

Proposed method

• Formulate MC acquisition functions as expectations over posterior samples and approximate with a fixed set of base samples (SAA).
• Use reparameterization to obtain differentiable sample paths and gradients for acquisition functions.
• Develop convergence guarantees for SAA optimization, including RQMC extensions.
• Introduce OKG, a deterministic, one-shot look-ahead KG formulation that avoids nested optimization by using fantasy models and a joint optimization over candidate points and fantasy points.
• Provide modular BoTorch abstractions that are model-agnostic and integrate with GPyTorch for fast predictive distributions and hardware acceleration.
• Demonstrate multi-objective and parallel/noisy acquisition functions through concrete implementations (e.g., qParEGO, qNEI, qKnowledgeGradient).

Experimental results

Research questions

• RQ1 Can MC acquisition functions be effectively optimized using a fixed set of base samples (SAA) with convergence guarantees?
• RQ2 Does a one-shot formulation of Knowledge Gradient (OKG) improve look-ahead BO performance and scalability compared to traditional KG methods?
• RQ3 How does BoTorch perform relative to other BO libraries in sample efficiency and computational efficiency across synthetic and real-world tasks?
• RQ4 Can BoTorch integrate parallel, asynchronous, multi-objective, and noisy acquisition functions within a differentiable, GPU-accelerated framework?

Key findings

• SAA-based optimization of MC acquisition functions converges to the true optimum with a.s. convergence and exponential convergence rates under mild conditions.
• RQMC-based methods further improve empirical convergence rates over MC in practice.
• OKG provides a scalable, one-shot look-ahead KG formulation with convergence guarantees and competitive asymptotic optimality.
• BoTorch demonstrates significant speedups and scalability on GPU and CPU, with 10–40X speedups from fast predictive distributions and near-linear scaling with batch size.
• OKG often outperforms other acquisition functions in synthetic tests and shows up to 6X wall-time reductions compared with MOE KG while delivering better optimization performance.
• Empirical results across Hartmann, Cartpole DQN tuning, and other hyperparameter optimization tasks show improved sample efficiency and practical performance.