[Paper Review] Parallel Bayesian Optimization of Multiple Noisy Objectives with Expected Hypervolume Improvement

Samuel Daulton, Maximilian Balandat+1 more|arXiv (Cornell University)|May 17, 2021

Advanced Multi-Objective Optimization AlgorithmsComputer Science92 citations

TL;DR

The paper introduces NEHVI for noisy multi-objective Bayesian optimization and a scalable parallel variant q NEHVI using cached box decompositions to handle large batches with noise, achieving state-of-the-art performance.

ABSTRACT

Optimizing multiple competing black-box objectives is a challenging problem in many fields, including science, engineering, and machine learning. Multi-objective Bayesian optimization (MOBO) is a sample-efficient approach for identifying the optimal trade-offs between the objectives. However, many existing methods perform poorly when the observations are corrupted by noise. We propose a novel acquisition function, NEHVI, that overcomes this important practical limitation by applying a Bayesian treatment to the popular expected hypervolume improvement (EHVI) criterion and integrating over this uncertainty in the Pareto frontier. We argue that, even in the noiseless setting, generating multiple candidates in parallel is an incarnation of EHVI with uncertainty in the Pareto frontier and therefore can be addressed using the same underlying technique. Through this lens, we derive a natural parallel variant, $q$NEHVI, that reduces computational complexity of parallel EHVI from exponential to polynomial with respect to the batch size. $q$NEHVI is one-step Bayes-optimal for hypervolume maximization in both noisy and noiseless environments, and we show that it can be optimized effectively with gradient-based methods via sample average approximation. Empirically, we demonstrate not only that $q$NEHVI is substantially more robust to observation noise than existing MOBO approaches, but also that it achieves state-of-the-art optimization performance and competitive wall-times in large-batch environments.

Motivation & Objective

• Identify optimal trade-offs (Pareto frontier) among multiple noisy black-box objectives with high sample efficiency.
• Develop an acquisition function that properly accounts for observation noise in MOBO.
• Enable scalable parallel/batch evaluations to improve throughput in real-world settings.
• Provide a theoretically sound and empirically robust method that outperforms baselines under noise.

Proposed method

• Define noisy expected hypervolume improvement (NEHVI) by integrating EHVI over the posterior p(f|D) of function values at observed points.
• Approximate NEHVI via Monte Carlo using samples from the posterior and joint samples with the candidate points (full-MC variant).
• Introduce Cached Box Decompositions (CBD) to reuse Pareto frontiers and box decompositions across evaluations, reducing time/space complexity from exponential to polynomial in batch size q.
• Extend to parallel/batch settings with q NEHVI using a sequential greedy approximation to maintain tractable optimization.
• Prove that NEHVI retains one-step Bayes-optimality in noisy environments and that CBD enables differentiable acquisition optimization with gradient-based methods.
• Offer a cheaper approximation q NEHVI-1 using a single GP sample path with random Fourier features for faster batch selection.

Experimental results

Research questions

• RQ1 How can MOBO be effectively performed when observations are noisy rather than noiseless?
• RQ2 Can EHVI-based acquisitions be scaled to large batch sizes in the presence of noise without sacrificing performance?
• RQ3 Does integrating over uncertainty in the Pareto frontier (NEHVI) improve robustness and diversity of the Pareto frontier under noisy observations?
• RQ4 How can we efficiently compute and optimize NEHVI in practice for large-scale problems?

Key findings

• NEHVI is more robust to observation noise than existing MOBO approaches, yielding well-distributed Pareto frontiers without clumping.
• The CBD approach reduces parallel EHVI’s time/space complexity from exponential to polynomial in batch size q, enabling large batches.
• q NEHVI achieves state-of-the-art optimization performance and competitive wall-times in large-batch settings.
• q NEHVI-1 offers a faster alternative with gradient-based optimization, scaling well on GPU hardware.
• Experiments show strong performance across synthetic and real-world noisy MOBO benchmarks against multiple baselines.