[Paper Review] Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery

Yuxin Wen, Neel Jain+4 more|arXiv (Cornell University)|Feb 7, 2023

Gene expression and cancer classificationBiochemistry, Genetics and Molecular Biology40 citations

TL;DR

The paper introduces PEZ, a gradient-based method to optimize discrete hard prompts by continuously updating soft prompts and projecting onto nearest vocabulary embeddings, enabling effective hard prompts for both image generation and language classification with transferability across models.

ABSTRACT

The strength of modern generative models lies in their ability to be controlled through text-based prompts. Typical "hard" prompts are made from interpretable words and tokens, and must be hand-crafted by humans. There are also "soft" prompts, which consist of continuous feature vectors. These can be discovered using powerful optimization methods, but they cannot be easily interpreted, re-used across models, or plugged into a text-based interface. We describe an approach to robustly optimize hard text prompts through efficient gradient-based optimization. Our approach automatically generates hard text-based prompts for both text-to-image and text-to-text applications. In the text-to-image setting, the method creates hard prompts for diffusion models, allowing API users to easily generate, discover, and mix and match image concepts without prior knowledge on how to prompt the model. In the text-to-text setting, we show that hard prompts can be automatically discovered that are effective in tuning LMs for classification.

Motivation & Objective

• Enable automatic learning of hard, human-readable prompts without hand-crafting.
• Bridge soft-prompt optimization with discrete hard prompts for portability and interpretability.
• Demonstrate effectiveness in image generation with diffusion models and language classification tasks.
• Show cross-model transferability of learned prompts and benefits of fluency constraints for interpretability.

Proposed method

• Maintain continuous prompt embeddings, projecting each embedding to the nearest token embedding to enforce discreteness.
• Update the continuous prompt via gradients computed on a task-specific loss using a broadcasted batch of the prompt.
• Optimize hard prompts with a gradient-based scheme inspired by discrete optimization and quantized-network literature.
• Apply a CLIP-based loss for image-to-text alignment to discover captions that guide diffusion models.
• Extend the approach to language models by combining task loss with a fluency penalty for interpretable prompts.

Experimental results

Research questions

• RQ1 Can gradient-based optimization reliably learn discrete hard prompts that are interpretable and transferable across models?
• RQ2 How do learned hard prompts perform for text-to-image generation and text-to-text classification compared to baselines?
• RQ3 What is the impact of prompt length and fluency constraints on performance and transferability?
• RQ4 Can prompts discovered from images via CLIP guidance be used directly to steer diffusion models effectively?
• RQ5 Do hard prompts maintain effectiveness when concatenated, distilled to shorter prompts, or used for style transfer?

Key findings

• Learned hard prompts (PEZ) achieve competitive CLIP-based similarity scores for image generation across multiple datasets, using far fewer tokens than some baselines.
• PEZ with a keyword bank or CLIP-based guidance can match or approach the performance of the CLIP Interrogator while using fewer tokens and no heavy auxiliary models.
• In language tasks, PEZ with or without fluency constraints yields competitive or superior accuracy transfer across GPT-2 family and larger LMs, outperforming several baselines on AGNEWS.
• Longer prompts can overfit and transfer less well than shorter, distilled prompts, with an empirically optimal length around 16 tokens for image generation tasks.
• Prompts learned via PEZ transfer better across models when fluency is enforced, and concatenation or distillation can compose or compress prompts without substantial loss of semantics.
• Safety concerns are discussed, including the potential for prompts to bypass content filters and reproduce banned content in certain APIs.