[Paper Review] Joint Entropy Search for Multi-objective Bayesian Optimization

Ben Tu, Axel Gandy+2 more|arXiv (Cornell University)|Oct 6, 2022

Advanced Multi-Objective Optimization Algorithms13 citations

TL;DR

This paper proposes Joint Entropy Search (JES), a novel information-theoretic acquisition function for multi-objective Bayesian optimization that maximizes joint information gain over both optimal inputs and outputs. By modeling uncertainty in the Pareto-optimal set and using analytically tractable approximations, JES improves sample efficiency and demonstrates superior performance on synthetic and real-world problems using hypervolume and weighted hypervolume metrics.

ABSTRACT

Many real-world problems can be phrased as a multi-objective optimization problem, where the goal is to identify the best set of compromises between the competing objectives. Multi-objective Bayesian optimization (BO) is a sample efficient strategy that can be deployed to solve these vector-valued optimization problems where access is limited to a number of noisy objective function evaluations. In this paper, we propose a novel information-theoretic acquisition function for BO called Joint Entropy Search (JES), which considers the joint information gain for the optimal set of inputs and outputs. We present several analytical approximations to the JES acquisition function and also introduce an extension to the batch setting. We showcase the effectiveness of this new approach on a range of synthetic and real-world problems in terms of the hypervolume and its weighted variants.

Motivation & Objective

• To address the challenge of sample-efficient multi-objective optimization where function evaluations are costly and noisy.
• To develop an acquisition function that captures joint uncertainty in both optimal inputs and outputs, rather than focusing on one or the other.
• To provide a principled, information-theoretic alternative to scalarization-based or improvement-based acquisition functions that may suffer from bias or poor exploration.
• To ensure invariance to objective reparameterization, making the method robust to scale and transformation of objectives.
• To extend the framework to batch, constrained, and decoupled evaluation settings for broader practical applicability.

Proposed method

• Proposes Joint Entropy Search (JES), an acquisition function that quantifies the expected reduction in entropy of the joint posterior distribution over optimal inputs and outputs.
• Derives analytical approximations to the JES acquisition function, including JES-LB2 and JES-0, using conditional entropy estimates and box decomposition techniques.
• Introduces a batch extension of JES by generalizing the information gain to multiple simultaneous evaluations.
• Adapts JES to constrained optimization by incorporating inequality constraints into the feasible region of the cumulative distribution function (CDF).
• Enables decoupled evaluations by showing that JES-LB2 and MES-LB2 are additive across objectives, allowing selective querying of individual objectives.
• Extends the framework to multi-fidelity settings by combining JES with cost-weighted fidelity modeling and conditional entropy decomposition.

Experimental results

Research questions

• RQ1 Can an acquisition function that jointly models uncertainty in both optimal inputs and outputs lead to better sample efficiency in multi-objective Bayesian optimization?
• RQ2 How does JES compare to scalarization-based and improvement-based acquisition functions in terms of convergence speed and hypervolume performance?
• RQ3 To what extent is JES invariant to reparameterization of objectives, and how does this affect robustness in high-dimensional or scale-mismatched problems?
• RQ4 Can JES be effectively extended to batch, constrained, and decoupled evaluation settings without sacrificing theoretical grounding?
• RQ5 Does the proposed weighted hypervolume metric provide a more nuanced assessment of multi-objective optimization performance than standard hypervolume?

Key findings

• JES achieves superior hypervolume improvement compared to state-of-the-art baselines on synthetic and real-world multi-objective problems, particularly in early iterations.
• The analytical approximations of JES—JES-LB2 and JES-0—are differentiable and computationally tractable, enabling efficient optimization via gradient-based methods.
• JES demonstrates invariance to objective reparameterization, making it robust to scaling and transformation of objectives, unlike many improvement-based metrics.
• The weighted hypervolume strategy proposed in the appendix enables more granular evaluation of performance across different regions of the objective space.
• The extension to constrained optimization is effective, with the feasible region decomposed into boxes to compute the constrained CDF and information gain.
• The decoupled evaluation variant of JES shows theoretical compatibility with selective objective querying, though empirical gains over full evaluations remain marginal in current benchmarks.