[Paper Review] Scalable Thompson Sampling using Sparse Gaussian Process Models

Sattar Vakili, Henry B. Moss+3 more|Lancaster EPrints (Lancaster University)|Jun 9, 2020

Advanced Bandit Algorithms Research72 references4 citations

TL;DR

This paper proposes scalable Thompson Sampling using sparse Gaussian processes (S-GP-TS), enabling efficient batch Bayesian optimization with large budgets. By leveraging sparse variational inference and decoupled sampling, it achieves computational complexity reductions without sacrificing regret performance, matching theoretical guarantees of exact GP-TS under approximation conditions.

ABSTRACT

Thompson Sampling (TS) from Gaussian Process (GP) models is a powerful tool for the optimization of black-box functions. Although TS enjoys strong theoretical guarantees and convincing empirical performance, it incurs a large computational overhead that scales polynomially with the optimization budget. Recently, scalable TS methods based on sparse GP models have been proposed to increase the scope of TS, enabling its application to problems that are sufficiently multi-modal, noisy or combinatorial to require more than a few hundred evaluations to be solved. However, the approximation error introduced by sparse GPs invalidates all existing regret bounds. In this work, we perform a theoretical and empirical analysis of scalable TS. We provide theoretical guarantees and show that the drastic reduction in computational complexity of scalable TS can be enjoyed without loss in the regret performance over the standard TS. These conceptual claims are validated for practical implementations of scalable TS on synthetic benchmarks and as part of a real-world high-throughput molecular design task.

Motivation & Objective

• Address the high computational cost of standard Thompson Sampling with Gaussian Processes, which scales cubically with observations.
• Enable large-batch Bayesian optimization for problems requiring hundreds or thousands of evaluations, such as noisy, multi-modal, or combinatorial tasks.
• Provide theoretical justification that scalable GP-TS maintains the same regret order as exact GP-TS under specific posterior approximation conditions.
• Validate the method empirically on synthetic benchmarks and a real-world high-throughput molecular design task.
• Demonstrate that sparse GP approximations with decoupled sampling do not degrade regret performance when approximation quality is sufficient.

Proposed method

• Use Sparse Variational Gaussian Processes (SVGP) to approximate the full GP posterior with a low-rank structure, reducing computational cost from O(t³) to O(m²t), where m ≪ t is the number of inducing variables.
• Apply decoupled sampling: decompose Thompson samples into a fixed prior component (based on M features) and a model update, enabling O((m+M)N) sampling complexity.
• Introduce theoretical conditions (Assumptions 3 and 4) under which approximate GP-TS achieves the same regret order as exact GP-TS.
• Leverage spectral bounds from prior work to derive requirements on the number of inducing variables and prior features to ensure low regret.
• Implement S-GP-TS with K-means and greedy variance-based inducing point selection, comparing their impact on performance.
• Use ADAM optimization with early stopping and learning rate scheduling for model training, and apply GIBBON as a baseline acquisition function in comparisons.

Experimental results

Research questions

• RQ1 Can Thompson Sampling with sparse Gaussian processes maintain the same regret order as exact GP-TS under controlled approximation errors?
• RQ2 What conditions on the posterior approximation quality are necessary to preserve regret performance in scalable TS?
• RQ3 How do different inducing point selection strategies (K-means vs. greedy variance) affect the performance of S-GP-TS on high-dimensional, noisy, or multi-modal functions?
• RQ4 To what extent can S-GP-TS outperform existing batch Bayesian optimization methods in terms of regret and iteration efficiency?
• RQ5 Can S-GP-TS scale effectively to real-world high-throughput molecular design tasks with large optimization budgets?

Key findings

• S-GP-TS achieves regret performance comparable to exact GP-TS when the posterior approximation satisfies specific theoretical conditions (Assumptions 3 and 4).
• On the 5D Ackley function with high noise (variance 0.5), S-GP-TS with B=100 outperforms sequential baselines and achieves lower regret in fewer iterations.
• For the noisy Hartmann function (6D), S-GP-TS with B=100 achieves lower regret than batch methods like qEI and LP-EI, which are limited to B=25.
• Greedy variance-based inducing point selection slightly outperforms K-means in high-exploration settings like the Ackley function, suggesting benefits for complex landscapes.
• In the idealized parallel setting (measured by BO iterations), S-GP-TS reaches low regret in a fraction of the iterations required by standard BO routines.
• The method successfully scales to a real-world high-throughput molecular design task, demonstrating practical viability beyond synthetic benchmarks.