[Paper Review] No-regret Algorithms for Multi-task Bayesian Optimization

Sayak Ray Chowdhury, Aditya Gopalan|arXiv (Cornell University)|Aug 20, 2020

Gaussian Processes and Bayesian Inference48 references4 citations

TL;DR

This paper proposes two novel multi-task Bayesian optimization algorithms, MT-KB and MT-BKB, that leverage multi-task kernels and random scalarizations to efficiently optimize vector-valued functions with shared structure. The authors establish the first finite-time regret bounds for multi-objective BO that explicitly capture inter-task similarity, demonstrating improved sample efficiency through theoretical guarantees and empirical validation on synthetic and real-world datasets.

ABSTRACT

We consider multi-objective optimization (MOO) of an unknown vector-valued function in the non-parametric Bayesian optimization (BO) setting, with the aim being to learn points on the Pareto front of the objectives. Most existing BO algorithms do not model the fact that the multiple objectives, or equivalently, tasks can share similarities, and even the few that do lack rigorous, finite-time regret guarantees that capture explicitly inter-task structure. In this work, we address this problem by modelling inter-task dependencies using a multi-task kernel and develop two novel BO algorithms based on random scalarizations of the objectives. Our algorithms employ vector-valued kernel regression as a stepping stone and belong to the upper confidence bound class of algorithms. Under a smoothness assumption that the unknown vector-valued function is an element of the reproducing kernel Hilbert space associated with the multi-task kernel, we derive worst-case regret bounds for our algorithms that explicitly capture the similarities between tasks. We numerically benchmark our algorithms on both synthetic and real-life MOO problems, and show the advantages offered by learning with multi-task kernels.

Motivation & Objective

• To address the lack of finite-time regret guarantees in multi-objective Bayesian optimization (MOO) that explicitly model inter-task dependencies.
• To develop algorithms that exploit similarities between multiple objectives to improve sample efficiency in black-box optimization.
• To provide theoretical regret bounds that reflect the degree of task similarity, rather than treating objectives independently.
• To integrate vector-valued kernel ridge regression and Nyström approximation into a UCB-style framework for scalable multi-task MOO.
• To empirically validate the proposed algorithms on synthetic and real-world MOO problems, showing advantages over independent and baseline methods.

Proposed method

• The algorithms use random scalarizations of the vector-valued objective to transform multi-objective optimization into a sequence of single-objective problems.
• A multi-task kernel is employed to model correlations across objectives, enabling shared learning across related tasks.
• Vector-valued kernel ridge regression is used as a surrogate model to estimate the joint objective function with uncertainty quantification.
• The algorithms follow the upper confidence bound (UCB) principle, selecting the next evaluation point based on a balance of predicted improvement and uncertainty.
• MT-BKB applies the Nyström approximation to reduce computational cost, enabling scalability to larger datasets.
• Theoretical analysis derives novel concentration inequalities for vector-valued function estimates, leading to regret bounds that depend on task similarity and kernel structure.

Experimental results

Research questions

• RQ1 Can multi-task Bayesian optimization with shared kernel structure achieve better regret performance than independent optimization in multi-objective settings?
• RQ2 How do inter-task similarities affect the finite-time regret of multi-objective BO algorithms?
• RQ3 Can random scalarizations be effectively combined with multi-task kernels to maintain flexibility in Pareto front exploration?
• RQ4 What is the impact of using the Nyström approximation on the regret and computational efficiency of multi-task BO?
• RQ5 Do the proposed algorithms achieve tighter regret bounds that explicitly reflect task similarity compared to existing methods?

Key findings

• The proposed MT-KB and MT-BKB algorithms achieve finite-time regret bounds that explicitly depend on the inter-task similarity structure through the multi-task kernel.
• The regret bounds are the first worst-case (frequentist) guarantees for multi-objective Bayesian optimization, derived using a novel concentration inequality for vector-valued function estimates.
• Empirical results show that MT-KB and MT-BKB outperform independent task baselines (IT-KB, IT-BKB) and MOBO in terms of both time-average cumulative regret and Bayes regret on synthetic and real sensor data.
• The use of the Nyström approximation in MT-BKB enables scalable inference while preserving theoretical guarantees, making it suitable for larger-scale problems.
• On real-world sensor data from the Intel Berkeley Lab, the algorithms demonstrate improved convergence and robustness compared to baselines, even without hyperparameter tuning.
• The algorithms are simple to implement when the kernel decomposes across tasks and input domain, and they maintain strong performance across diverse problem types including RKHS and perturbed sine functions.