[Paper Review] SPELL: Semantic Prompt Evolution based on a LLM

Yujian Li, Kai Wu|arXiv (Cornell University)|Oct 2, 2023

Topic Modeling5 citations

TL;DR

SPELL proposes a black-box prompt optimization method that leverages a large language model (LLM) as a semantic text generator to evolve coherent, high-performing prompts through evolutionary computation. By using the LLM to generate fluent, globally optimized prompts via reproduction and selection, SPELL achieves significant performance gains—e.g., 89.2% accuracy on SST-2—while maintaining fluency, outperforming other black-box methods in fewer rounds.

ABSTRACT

Prompt engineering is a new paradigm for enhancing the performance of trained neural network models. For optimizing text-style prompts, existing methods usually individually operate small portions of a text step by step, which either breaks the fluency or could not globally adjust a prompt. Since large language models (LLMs) have powerful ability of generating coherent texts token by token, can we utilize LLMs for improving prompts? Based on this motivation, in this paper, considering a trained LLM as a text generator, we attempt to design a black-box evolution algorithm for automatically optimizing texts, namely SPELL (Semantic Prompt Evolution based on a LLM). The proposed method is evaluated with different LLMs and evolution parameters in different text tasks. Experimental results show that SPELL could rapidly improve the prompts indeed. We further explore the evolution process and discuss on the limitations, potential possibilities and future work.

Motivation & Objective

• To address the limitations of existing prompt optimization methods that modify prompts locally and degrade fluency.
• To explore whether large language models (LLMs) can serve as effective generators for evolving high-quality, coherent prompts.
• To develop a black-box evolutionary framework that optimizes prompts without gradient access to the target model.
• To evaluate the impact of different LLMs and evolution hyperparameters on prompt quality and task performance.
• To analyze the stability, coherence, and scalability of the proposed semantic prompt evolution process.

Proposed method

• SPELL maintains a population of prompt individuals and evolves them over generations using reproduction and selection mechanisms.
• Reproduction generates offspring by prompting a pre-trained LLM to rewrite or improve existing prompts based on fitness feedback.
• The LLM acts as a text generator, ensuring semantic coherence and global optimization by modeling the entire prompt context token-by-token.
• Selection retains the fittest individuals from the union of parents and offspring, using accuracy on a validation set as the fitness metric.
• The method operates in a black-box setting, requiring only inference access to the target model and no gradient or parameter access.
• Evolution proceeds over T generations, with hyperparameters like population size (N_POP), selection k, and fitness function tuned for balance between exploration and exploitation.

Experimental results

Research questions

• RQ1 Can a large language model be effectively used as a semantic generator to evolve coherent and high-performing prompts?
• RQ2 How does SPELL compare to existing black-box prompt optimization methods in terms of performance and convergence speed?
• RQ3 What impact do different LLMs used as generators have on the quality and stability of evolved prompts?
• RQ4 How do key hyperparameters—such as population size and selection k—affect the optimization outcome?
• RQ5 What are the limitations and challenges of using LLMs for semantic prompt evolution, particularly regarding instability and coherence?

Key findings

• SPELL achieved 89.2% accuracy on the SST-2 benchmark, outperforming other black-box methods like BBT (88.2%) and RLPrompt (90.5%) in zero-shot settings.
• The method required significantly fewer optimization rounds than baseline black-box methods—e.g., fewer than 3k rounds for 5-token prompts—demonstrating rapid convergence.
• Using the target model’s accuracy as the fitness function yielded better results than using cross-entropy loss, suggesting that LLMs may not effectively interpret loss signals.
• Larger and more capable LLMs as generators, such as ERNIE-Bot and Llama2-13B, produced higher-performing prompts than smaller or less aligned models like BLOOMZ.
• Population size and selection k had a measurable impact: too small or too large populations reduced performance, with optimal balance found at N_POP=20 and k=16.
• Despite performance fluctuations due to randomness in generation, evolved prompts remained semantically coherent and fluent, as demonstrated by examples like 'Can you determine the sentiment of this sentence for me?'