[Paper Review] Bandit Theory and Thompson Sampling-Guided Directed Evolution for Sequence Optimization

Hui Yuan, Chengzhuo Ni+5 more|arXiv (Cornell University)|Jun 5, 2022

Advanced Multi-Objective Optimization Algorithms4 citations

TL;DR

This paper proposes Thompson Sampling-guided Directed Evolution (TS-DE), a Bayesian bandit framework that integrates posterior sampling with evolutionary computation to optimize biological sequences under costly, noisy measurements. For linear sequence-to-function mappings, it achieves a Bayesian regret bound of $\widetilde{O}(d^2\sqrt{MT})$, demonstrating theoretical acceleration of directed evolution through active learning principles.

ABSTRACT

Directed Evolution (DE), a landmark wet-lab method originated in 1960s, enables discovery of novel protein designs via evolving a population of candidate sequences. Recent advances in biotechnology has made it possible to collect high-throughput data, allowing the use of machine learning to map out a protein's sequence-to-function relation. There is a growing interest in machine learning-assisted DE for accelerating protein optimization. Yet the theoretical understanding of DE, as well as the use of machine learning in DE, remains limited. In this paper, we connect DE with the bandit learning theory and make a first attempt to study regret minimization in DE. We propose a Thompson Sampling-guided Directed Evolution (TS-DE) framework for sequence optimization, where the sequence-to-function mapping is unknown and querying a single value is subject to costly and noisy measurements. TS-DE updates a posterior of the function based on collected measurements. It uses a posterior-sampled function estimate to guide the crossover recombination and mutation steps in DE. In the case of a linear model, we show that TS-DE enjoys a Bayesian regret of order $ ilde O(d^{2}\sqrt{MT})$, where $d$ is feature dimension, $M$ is population size and $T$ is number of rounds. This regret bound is nearly optimal, confirming that bandit learning can provably accelerate DE. It may have implications for more general sequence optimization and evolutionary algorithms.

Motivation & Objective

• To formalize directed evolution (DE) as a bandit optimization problem with costly, noisy function evaluations.
• To develop a machine learning-augmented DE framework that reduces experimental cost through active, Bayesian exploration.
• To establish a theoretical regret bound for sequence optimization under unknown sequence-to-function mappings.
• To validate the framework using both simulated and real-world biological sequence optimization tasks.

Proposed method

• Proposes a Thompson Sampling-guided Directed Evolution (TS-DE) framework that uses posterior sampling to guide crossover and mutation in evolutionary operations.
• Maintains a Gaussian process posterior over the unknown sequence-to-function mapping, updated with each new measurement.
• Uses sampled function estimates from the posterior to rank and select high-potential sequences for recombination and mutation.
• Employs confidence ellipsoids and sequential Bayesian inference to bound estimation error and control exploration-exploitation trade-off.
• Derives a regret bound using concentration inequalities and properties of Gaussian processes under linear models.
• Integrates population diversity into the optimization process, where crossover selection acts as a form of iterative improvement.

Experimental results

Research questions

• RQ1 Can directed evolution be formally framed as a Bayesian bandit problem with regret minimization guarantees?
• RQ2 How does Thompson Sampling improve the efficiency of sequence optimization under costly and noisy measurements?
• RQ3 What is the theoretical regret bound of a machine learning-augmented DE algorithm under a linear model assumption?
• RQ4 How does population size and feature dimension affect the convergence rate of the optimization process?
• RQ5 Can the theoretical performance be validated on both synthetic and real biological sequence datasets?

Key findings

• The TS-DE framework achieves a Bayesian regret bound of $\widetilde{O}(d^2\sqrt{MT})$ for linear sequence-to-function mappings, where $d$ is the feature dimension, $M$ is the population size, and $T$ is the number of rounds.
• The regret bound is nearly optimal, confirming that bandit learning can provably accelerate directed evolution.
• The theoretical analysis shows that population diversity and crossover selection are critical for optimization progress in the TS-DE framework.
• The posterior sampling mechanism effectively balances exploration and exploitation, reducing the number of costly experimental queries.
• Empirical validation on simulation and real-world data confirms the alignment between theoretical predictions and practical performance.
• The framework provides a principled, theoretically grounded approach to accelerating protein and sequence design using machine learning.