[Paper Review] BOSH: Bayesian Optimization by Sampling Hierarchically

Henry B. Moss, David S. Leslie+1 more|arXiv (Cornell University)|Jul 2, 2020

Gaussian Processes and Bayesian Inference32 references4 citations

TL;DR

BOSH introduces a novel Bayesian optimization framework that dynamically grows a pool of stochastic realizations using a hierarchical Gaussian process and an information-theoretic acquisition strategy, enabling more efficient and precise optimization than standard BO across synthetic, simulation, reinforcement learning, and hyperparameter tuning tasks by adaptively balancing evaluation of existing realizations and sampling new ones.

ABSTRACT

Deployments of Bayesian Optimization (BO) for functions with stochastic evaluations, such as parameter tuning via cross validation and simulation optimization, typically optimize an average of a fixed set of noisy realizations of the objective function. However, disregarding the true objective function in this manner finds a high-precision optimum of the wrong function. To solve this problem, we propose Bayesian Optimization by Sampling Hierarchically (BOSH), a novel BO routine pairing a hierarchical Gaussian process with an information-theoretic framework to generate a growing pool of realizations as the optimization progresses. We demonstrate that BOSH provides more efficient and higher-precision optimization than standard BO across synthetic benchmarks, simulation optimization, reinforcement learning and hyper-parameter tuning tasks.

Motivation & Objective

• To address the limitation of standard Bayesian optimization in stochastic settings, where fixed evaluation strategies (e.g., K-fold CV) optimize an average of noisy realizations rather than the true objective function.
• To reduce the risk of converging to a high-precision optimum of a wrong function due to suboptimal or fixed realization sets.
• To develop a method that adaptively grows a pool of realizations during optimization, improving both efficiency and precision without predefining the number of realizations.
• To enable batch acquisition and scalable computation in stochastic optimization, overcoming limitations of existing multi-fidelity and multi-task BO frameworks.
• To provide a principled, information-theoretic framework for deciding when to evaluate existing realizations versus sample new ones.

Proposed method

• BOSH employs a Hierarchical Gaussian Process (HGP) to model the true objective function as a latent function, with individual realizations as perturbations of this latent function.
• The HGP uses shared length-scales across kernels to improve model stability and enable reliable initialization with only d+5 evaluations.
• An information-theoretic acquisition function quantifies the utility of evaluating a new realization or an existing one, balancing exploration of new realizations and exploitation of existing ones.
• The framework supports batch acquisition by extending the acquisition function to recommend multiple points across realizations, enabling parallel evaluation.
• The method avoids the need for fixed evaluation strategies by dynamically expanding the realization pool as optimization progresses.
• The HGP is trained on a growing set of evaluations from multiple realizations, with uncertainty propagated through the hierarchy to guide decisions.

Experimental results

Research questions

• RQ1 Can a Bayesian optimization framework dynamically grow a pool of realizations during optimization to improve precision compared to fixed evaluation strategies?
• RQ2 Does an information-theoretic acquisition function that balances new realization sampling and existing realization evaluation lead to more efficient optimization?
• RQ3 Can BOSH achieve higher-precision optimization than standard BO and existing multi-fidelity or multi-task BO methods in simulation, reinforcement learning, and hyperparameter tuning?
• RQ4 How does BOSH’s adaptive strategy compare to fixed-K approaches in terms of initialization cost and convergence speed?
• RQ5 To what extent can BOSH recommend effective batches of evaluations while maintaining high-precision optimization?

Key findings

• BOSH achieved higher-precision optimization than standard BO across all benchmark tasks, including synthetic, simulation, reinforcement learning, and hyperparameter tuning.
• In the Lunar Lander task, BOSH matched the performance of the hard-coded PID controller using only 10 parallel evaluations, outperforming standard BO and batch BO.
• For IMDB movie review classification, BOSH used up to four train-test splits to achieve more precise SVM hyperparameter tuning than standard BO with K-fold CV.
• In the warehouse allocation simulation, BOSH achieved high-precision optimization where standard BO only provided rough optimization, even with parallel evaluations.
• BOSH required significantly lower initialization costs than FASTCV, which needed d+3 evaluations per realization, while BOSH initialized reliably with d+5 total evaluations across two realizations.
• BOSH’s adaptive strategy enabled faster and more precise tuning than batch BO under fixed evaluation strategies, especially when parallel resources were available.