[Paper Review] A New Knowledge Gradient-based Method for Constrained Bayesian Optimization

Wenjie Chen, Shengcai Liu+1 more|arXiv (Cornell University)|Jan 20, 2021

Advanced Multi-Objective Optimization Algorithms33 references6 citations

TL;DR

This paper proposes a novel constrained Bayesian optimization method, c-KG, which extends the knowledge gradient framework to handle noisy, black-box objective and constraint functions with expensive evaluations. By introducing a new acquisition function that balances optimality and feasibility, and using an unbiased gradient estimator via IPA and LR, c-KG enables efficient batch sampling for constrained optimization under uncertainty.

ABSTRACT

Black-box problems are common in real life like structural design, drug experiments, and machine learning. When optimizing black-box systems, decision-makers always consider multiple performances and give the final decision by comprehensive evaluations. Motivated by such practical needs, we focus on constrained black-box problems where the objective and constraints lack known special structure, and evaluations are expensive and even with noise. We develop a novel constrained Bayesian optimization approach based on the knowledge gradient method ($c- m{KG}$). A new acquisition function is proposed to determine the next batch of samples considering optimality and feasibility. An unbiased estimator of the gradient of the new acquisition function is derived to implement the $c- m{KG}$ approach.

Motivation & Objective

• Address the challenge of optimizing black-box systems with multiple performance indicators and stochastic constraints, where evaluations are expensive and noisy.
• Develop a new acquisition function that integrates feasibility and optimality in constrained Bayesian optimization, extending the knowledge gradient method to handle noisy constraints.
• Enable efficient batch sampling for constrained optimization by formulating an inner optimization problem over the c-KG acquisition function.
• Provide an unbiased gradient estimator for the c-KG acquisition function using a hybrid of infinitesimal perturbation analysis (IPA) and likelihood ratio (LR) methods to support stochastic optimization.
• Improve convergence and feasibility in real-world applications such as drug discovery, materials design, and simulation-based system optimization under uncertainty.

Proposed method

• Propose a new acquisition function, c-KG, which generalizes the knowledge gradient to constrained problems by incorporating the probability of feasibility and expected improvement.
• Formulate the acquisition function as the expected improvement in the optimal objective value, conditioned on future observations, while accounting for constraint satisfaction.
• Derive an unbiased gradient estimator for c-KG using a hybrid of IPA and LR methods, enabling stochastic gradient ascent for batch selection.
• Use a multi-start stochastic gradient ascent algorithm to solve the inner optimization problem of maximizing c-KG over a batch of candidate points.
• Integrate Gaussian process priors for the objective and constraint functions, updating posteriors sequentially with observed data.
• Implement a framework where the best feasible solution is estimated at each iteration via constrained nonlinear programming on the GP posterior means.

Experimental results

Research questions

• RQ1 Can the knowledge gradient principle be effectively extended to handle stochastic constraints in black-box optimization with expensive, noisy evaluations?
• RQ2 How can an acquisition function be designed to simultaneously balance optimality and feasibility in constrained Bayesian optimization?
• RQ3 What is an efficient and unbiased way to estimate the gradient of a complex acquisition function involving indicator functions and Gaussian process posteriors?
• RQ4 How does the proposed c-KG method compare in convergence and feasibility to existing constrained Bayesian optimization approaches under noisy conditions?
• RQ5 Can batch sampling be effectively enabled through a tractable and differentiable acquisition function that accounts for both objective and constraint uncertainty?

Key findings

• The proposed c-KG acquisition function successfully integrates feasibility and optimality into a single, tractable objective, enabling effective exploration of the constrained search space.
• The hybrid IPA-LR gradient estimator provides an unbiased estimate of the c-KG gradient, allowing reliable stochastic optimization of the acquisition function.
• The method demonstrates improved convergence and feasibility in constrained optimization tasks, particularly in settings with noisy evaluations and no known functional structure.
• Empirical results show that c-KG outperforms baseline methods in finding feasible, high-quality solutions with fewer function evaluations, especially in multi-constraint and high-noise environments.
• The algorithm framework supports batch sampling, enabling parallel evaluation of multiple candidates, which is crucial for expensive real-world applications.
• Theoretical analysis confirms that the gradient estimator satisfies the necessary conditions for unbiasedness and convergence under mild regularity assumptions.