[Paper Review] Reducing Retraining by Recycling Parameter-Efficient Prompts

Brian Lester, Joshua Yurtsever+2 more|arXiv (Cornell University)|Aug 10, 2022

Topic Modeling6 citations

TL;DR

This paper proposes Prompt Recycling, a method to transfer soft prompts trained on a source frozen large language model (LLM) to a new target LLM without retraining, using structural correspondences between model embeddings. The best method achieves 88.9% success in recycling prompts that outperform the target model’s zero-shot performance, though performance still lags behind prompts fine-tuned from scratch.

ABSTRACT

Parameter-efficient methods are able to use a single frozen pre-trained large language model (LLM) to perform many tasks by learning task-specific soft prompts that modulate model behavior when concatenated to the input text. However, these learned prompts are tightly coupled to a given frozen model -- if the model is updated, corresponding new prompts need to be obtained. In this work, we propose and investigate several approaches to "Prompt Recycling'" where a prompt trained on a source model is transformed to work with the new target model. Our methods do not rely on supervised pairs of prompts, task-specific data, or training updates with the target model, which would be just as costly as re-tuning prompts with the target model from scratch. We show that recycling between models is possible (our best settings are able to successfully recycle $88.9\%$ of prompts, producing a prompt that out-performs baselines), but significant performance headroom remains, requiring improved recycling techniques.

Motivation & Objective

• To address the inefficiency of retraining soft prompts whenever a frozen LLM is updated, which is costly and time-consuming.
• To enable transfer of task-specific prompts from a source LLM to a new target LLM without requiring task-specific data or model updates.
• To explore whether structural similarities between models can be leveraged to transform prompts across models while preserving task performance.
• To evaluate the feasibility and effectiveness of prompt recycling across different model sizes and architectures.

Proposed method

• The method uses learned transformations based on the correspondence between word embeddings in the source and target models to map a soft prompt from the source model to a compatible prompt for the target model.
• It relies on the structural similarity of attention layers and embedding spaces between models, particularly focusing on how token representations align across models.
• Three recycling techniques are proposed: linear projection, residual adapter, and a method using class-specific initialization and SPoT (Sparse Prompt Tuning) to improve alignment.
• The approach does not require paired prompts, task-specific data, or fine-tuning the target model, preserving computational efficiency.
• Performance is evaluated by comparing recycled prompts to zero-shot and random prompt baselines on downstream tasks like SST-2 and IMDB.
• The method is tested across different model sizes, including recycling from larger to smaller models (Large → Base) and vice versa (Base → Large).

Experimental results

Research questions

• RQ1 Can soft prompts trained on a source LLM be effectively transferred to a new target LLM without retraining?
• RQ2 Does the performance of recycled prompts exceed the zero-shot capabilities of the target model?
• RQ3 How does the size of the source and target models affect the reliability and performance of prompt recycling?
• RQ4 Can structural correspondences between models be used to create effective prompt transformations without task-specific data?
• RQ5 Is prompt recycling more effective when transferring from larger to smaller models, or vice versa?

Key findings

• The best recycling method successfully recycles 88.9% of prompts in a way that outperforms the target model’s zero-shot performance.
• Recycling from a larger source model (Large) to a smaller target model (Base) yields more reliable and higher-performing prompts than recycling between models of the same size.
• Recycling from Base to Large is unstable, with high variance and often worse performance than random prompts, especially on IMDB.
• Prompts recycled from Large to Base show stronger performance than both zero-shot and random prompt baselines, suggesting potential for knowledge distillation-like transfer.
• Performance of recycled prompts remains significantly below that of prompts fine-tuned from scratch on the target model, indicating substantial headroom for improvement.
• The success of recycling is correlated with the prompt’s training regime—prompts trained on larger models tend to be more recyclable, possibly due to better generalization or knowledge encoding.