[Paper Review] Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization

Samuel Daulton, Xingchen Wan+4 more|arXiv (Cornell University)|Oct 18, 2022

Advanced Multi-Objective Optimization Algorithms16 citations

TL;DR

This paper proposes probabilistic reparameterization (PR), a method for Bayesian optimization over discrete and mixed spaces by reparameterizing discrete parameters as continuous probability distributions. By optimizing the expected acquisition function over these distributions, PR enables gradient-based optimization, achieves convergence to stationary points, and matches standard BO regret bounds, outperforming existing methods on real-world problems with discrete and mixed search spaces.

ABSTRACT

Optimizing expensive-to-evaluate black-box functions of discrete (and potentially continuous) design parameters is a ubiquitous problem in scientific and engineering applications. Bayesian optimization (BO) is a popular, sample-efficient method that leverages a probabilistic surrogate model and an acquisition function (AF) to select promising designs to evaluate. However, maximizing the AF over mixed or high-cardinality discrete search spaces is challenging standard gradient-based methods cannot be used directly or evaluating the AF at every point in the search space would be computationally prohibitive. To address this issue, we propose using probabilistic reparameterization (PR). Instead of directly optimizing the AF over the search space containing discrete parameters, we instead maximize the expectation of the AF over a probability distribution defined by continuous parameters. We prove that under suitable reparameterizations, the BO policy that maximizes the probabilistic objective is the same as that which maximizes the AF, and therefore, PR enjoys the same regret bounds as the original BO policy using the underlying AF. Moreover, our approach provably converges to a stationary point of the probabilistic objective under gradient ascent using scalable, unbiased estimators of both the probabilistic objective and its gradient. Therefore, as the number of starting points and gradient steps increase, our approach will recover of a maximizer of the AF (an often-neglected requisite for commonly used BO regret bounds). We validate our approach empirically and demonstrate state-of-the-art optimization performance on a wide range of real-world applications. PR is complementary to (and benefits) recent work and naturally generalizes to settings with multiple objectives and black-box constraints.

Motivation & Objective

• To address the challenge of optimizing acquisition functions in Bayesian optimization over high-cardinality or mixed discrete-continuous search spaces where gradient methods fail.
• To develop a method that enables efficient, scalable optimization of acquisition functions without exhaustive enumeration of discrete configurations.
• To prove that maximizing the probabilistic objective via reparameterization yields the same optimal policy as maximizing the original acquisition function, preserving theoretical regret bounds.
• To demonstrate empirically that the approach achieves state-of-the-art performance on real-world black-box optimization problems with discrete and mixed parameters.
• To ensure compatibility with existing BO frameworks, including multi-objective, constrained, and trust-region methods, by remaining agnostic to the choice of kernel over discrete variables.

Proposed method

• Reparameterize discrete parameters as continuous probability distributions, transforming the discrete optimization problem into a continuous one over the simplex of probabilities.
• Define a probabilistic objective (PO) as the expectation of the acquisition function over the reparameterized discrete distributions.
• Derive unbiased, scalable Monte Carlo estimators for both the probabilistic objective and its gradient with respect to the probability parameters.
• Apply stochastic gradient ascent using these estimators, which provably converges to a stationary point of the PO surface.
• Prove that the maximizers of the PO correspond exactly to the maximizers of the original acquisition function, ensuring equivalence in policy and regret guarantees.
• Integrate the method with standard Bayesian optimization pipelines, supporting arbitrary kernels over discrete variables, including categorical, embedding-based, and one-hot encoded representations.

Experimental results

Research questions

• RQ1 Can we reformulate acquisition function maximization over discrete and mixed spaces as a continuous optimization problem using probabilistic reparameterization?
• RQ2 Does optimizing the expected acquisition function over a probability distribution of discrete choices preserve the theoretical regret bounds of standard Bayesian optimization?
• RQ3 Can stochastic gradient ascent on the reparameterized objective converge to a global maximizer of the original acquisition function as the number of iterations and starting points increases?
• RQ4 How does the performance of probabilistic reparameterization compare to existing methods such as evolutionary algorithms and gradient-free optimization in real-world discrete and mixed optimization tasks?
• RQ5 Is the proposed method compatible with a wide range of probabilistic models and kernels over discrete parameters, including non-differentiable or discrete-input kernels?

Key findings

• Probabilistic reparameterization enables gradient-based optimization over discrete and mixed spaces, overcoming the incompatibility of standard gradient methods with discrete variables.
• The method provably converges to a stationary point of the probabilistic objective, and under sufficient iterations and starting points, recovers a global maximizer of the original acquisition function.
• Empirical results show that PR achieves state-of-the-art optimization performance across diverse real-world problems, including welded beam design and chemical reaction yield optimization.
• PR significantly outperforms the evolutionary algorithm PortfolioDiscreteOnePlusOne in terms of log regret, though it is slower in wall time.
• The approach is agnostic to the choice of kernel over discrete parameters, supporting categorical, embedding-based, and one-hot encoded kernels, making it broadly applicable.
• Stochastic optimization with Adam and Monte Carlo gradient estimators consistently outperforms deterministic optimization and stochastic gradient ascent with learning rate decay.