[Paper Review] Differentiable Expected Hypervolume Improvement for Parallel Multi-Objective Bayesian Optimization

Samuel Daulton, Maximilian Balandat+1 more|arXiv (Cornell University)|Jun 9, 2020

Advanced Multi-Objective Optimization Algorithms65 references121 citations

TL;DR

Introduces q-EHVI, a differentiable, parallel, multi-objective Bayesian optimization acquisition function that exactly computes joint EHVI up to MC error, with exact gradients via auto-differentiation and GPU acceleration, demonstrating superior performance and tractability.

ABSTRACT

In many real-world scenarios, decision makers seek to efficiently optimize multiple competing objectives in a sample-efficient fashion. Multi-objective Bayesian optimization (BO) is a common approach, but many of the best-performing acquisition functions do not have known analytic gradients and suffer from high computational overhead. We leverage recent advances in programming models and hardware acceleration for multi-objective BO using Expected Hypervolume Improvement (EHVI)---an algorithm notorious for its high computational complexity. We derive a novel formulation of q-Expected Hypervolume Improvement (qEHVI), an acquisition function that extends EHVI to the parallel, constrained evaluation setting. qEHVI is an exact computation of the joint EHVI of q new candidate points (up to Monte-Carlo (MC) integration error). Whereas previous EHVI formulations rely on gradient-free acquisition optimization or approximated gradients, we compute exact gradients of the MC estimator via auto-differentiation, thereby enabling efficient and effective optimization using first-order and quasi-second-order methods. Our empirical evaluation demonstrates that qEHVI is computationally tractable in many practical scenarios and outperforms state-of-the-art multi-objective BO algorithms at a fraction of their wall time.

Motivation & Objective

• Address sample-efficient optimization of multiple expensive objectives.
• Develop a differentiable parallel acquisition function for MO BO.
• Enable exact gradient computation of q-EHVI via auto-differentiation.
• Provide theoretical convergence guarantees under the SAA framework.
• Demonstrate empirical superiority and practical tractability on benchmarks and real-world problems.

Proposed method

• Formulate q-EHVI as the exact MC-based joint hypervolume improvement for q new points.
• Use box decompositions and inclusion-exclusion to compute HVI across non-dominated space.
• ComputeMC-estimator gradients exactly via re-parameterization and auto-differentiation.
• Apply Sample Average Approximation to enable deterministic higher-order optimizers for acquisition maximization.
• Extend to outcome constraints by feasibility-weighting via a differentiable sigmoid.
• Leverage GPU parallelism to achieve fast wall times and near-constant per-point computation in practice.

Experimental results

Research questions

• RQ1 Can q-EHVI provide exact joint EHVI for parallel MO BO up to MC error across M>2 objectives?
• RQ2 Do exact MC gradients enable efficient gradient-based optimization for q-EHVI compared to gradient-free methods?
• RQ3 How does q-EHVI perform with auxiliary outcome constraints and in high-parallelism settings?
• RQ4 What are the convergence guarantees when optimizing q-EHVI under the Sample Average Approximation framework?

Key findings

• q-EHVI enables exact gradients of the MC estimator for parallel MO BO with M>2 and constraints.
• The method is empirically faster than state-of-the-art MO BO baselines while often achieving better hypervolume coverage.
• Optimization with exact gradients substantially outperforms gradient-free and finite-difference approaches.
• Sequential greedy and joint optimization are both feasible, with sequential integration over pending points showing favorable performance.
• The approach scales efficiently on modern GPUs, delivering practical wall times and remaining robust as batch size q grows (subject to partitioning limits).
• Experiments on synthetic and real-world MO problems (including vehicle safety design and ABR control) show superior performance in terms of convergence speed and final hypervolume.