[Paper Review] Large Language Models As Evolution Strategies

Robert Lange, Yingtao Tian+1 more|arXiv (Cornell University)|Feb 28, 2024

Language and cultural evolution4 citations

TL;DR

This paper proposes EvoLLM, a zero-shot black-box optimization framework that uses large language models (LLMs) as recombination operators in evolution strategies by prompting them with discretized, fitness-sorted solution candidates. The method outperforms random search and Gaussian hill climbing on BBOB functions and neuroevolution tasks, with performance further enhanced via instruction fine-tuning on teacher algorithm trajectories.

ABSTRACT

Large Transformer models are capable of implementing a plethora of so-called in-context learning algorithms. These include gradient descent, classification, sequence completion, transformation, and improvement. In this work, we investigate whether large language models (LLMs), which never explicitly encountered the task of black-box optimization, are in principle capable of implementing evolutionary optimization algorithms. While previous works have solely focused on language-based task specification, we move forward and focus on the zero-shot application of LLMs to black-box optimization. We introduce a novel prompting strategy, consisting of least-to-most sorting of discretized population members and querying the LLM to propose an improvement to the mean statistic, i.e. perform a type of black-box recombination operation. Empirically, we find that our setup allows the user to obtain an LLM-based evolution strategy, which we call `EvoLLM', that robustly outperforms baseline algorithms such as random search and Gaussian Hill Climbing on synthetic BBOB functions as well as small neuroevolution tasks. Hence, LLMs can act as `plug-in' in-context recombination operators. We provide several comparative studies of the LLM's model size, prompt strategy, and context construction. Finally, we show that one can flexibly improve EvoLLM's performance by providing teacher algorithm information via instruction fine-tuning on previously collected teacher optimization trajectories.

Motivation & Objective

• To investigate whether LLMs can perform zero-shot black-box optimization without explicit training on optimization tasks.
• To design a prompting strategy that enables LLMs to act as recombination operators in evolution strategies using only text-based context.
• To evaluate the robustness and performance of LLM-based optimization across diverse synthetic and neuroevolution tasks.
• To explore whether instruction fine-tuning on teacher algorithm trajectories improves LLM-based optimization performance.
• To identify key design choices—such as solution representation, discretization, and context construction—that influence optimization success.

Proposed method

• The method uses a least-to-most sorted sequence of discretized solution candidates and their fitness values as input context to the LLM.
• A fitness improvement query is posed to the LLM, asking it to propose a new mean statistic for the next evolution strategy sampling distribution.
• Solutions are represented as integers via discretization of the continuous search space, with resolution tuned per problem.
• The approach supports block-wise processing to manage context length growth across high-dimensional problems.
• Performance is evaluated on BBOB functions and small neuroevolution control tasks using multiple LLMs and ablation studies.
• Instruction fine-tuning is applied using teacher algorithm trajectories (e.g., Gaussian Hill Climbing) to improve LLM performance.

Figure 1 . Left. Overview of EvoLLM procedure. An LLM proposes an ES search distribution update using a discretized search space & solutions sorted by their performance (least-to-most). To combat the context length growth in the number of dimensions, we can split the search dimensions into blocks an

Experimental results

Research questions

• RQ1 Can LLMs perform zero-shot black-box optimization without explicit training on optimization tasks?
• RQ2 Does a discretized, sorted, and query-based prompting strategy enable LLMs to act as effective recombination operators in evolution strategies?
• RQ3 How do model size, solution representation, and context construction affect the optimization performance of LLM-based evolution strategies?
• RQ4 Can instruction fine-tuning on trajectories from teacher algorithms improve the performance of LLM-based optimizers?
• RQ5 Is there an optimal discretization resolution for LLM-based optimization, and does it vary across problem types?

Key findings

• EvoLLM outperforms random search and Gaussian Hill Climbing on all 8 BBOB test functions and 3 neuroevolution control tasks, with consistent improvements across multiple runs.
• Smaller LLMs (e.g., PaLM-XS) outperform larger ones in most cases, suggesting that model size is not the primary determinant of performance.
• Discretized integer representation of solutions significantly outperforms natural language-based instructions, highlighting the importance of solution representation.
• Optimal discretization resolution is problem-specific: performance peaks at 1000 bins for some functions (e.g., Sphere), but degrades at 10,000 bins due to context overload.
• Instruction fine-tuning on trajectories from a Gaussian Hill Climbing teacher algorithm leads to a small but robust performance gain across unseen BBOB and neuroevolution tasks.
• Even with limited context (few members or generations), the LLM can infer and propose beneficial mean updates, indicating strong pattern generalization from sorted sequences.

Figure 2 . EvoLLM prompt design space & API. We track all solution evaluations and their performance in a context buffer. The buffer is used to construct query prompts for the LLM. After parsing the LLM output and sampling, we evaluate the resulting population and add the new information to the buff