[Paper Review] READ: Recurrent Adaptation of Large Transformers

John Nguyen, Sid Wang+2 more|arXiv (Cornell University)|May 24, 2023

Topic Modeling4 citations

TL;DR

READ introduces a lightweight, memory-efficient fine-tuning method for large Transformers by inserting a small RNN-based side network that adapts to task-specific inputs without backpropagating through the frozen backbone. It achieves 56% lower training memory and 84% reduced GPU energy usage compared to full fine-tuning, with no increase in model size or inference latency.

ABSTRACT

Fine-tuning large-scale Transformers has led to the explosion of many AI applications across Natural Language Processing and Computer Vision tasks. However, fine-tuning all pre-trained model parameters becomes impractical as the model size and number of tasks increase. Parameter-efficient transfer learning (PETL) methods aim to address these challenges. While effective in reducing the number of trainable parameters, PETL methods still require significant energy and computational resources to fine-tune. In this paper, we introduce extbf{RE}current extbf{AD}aption (READ) -- a lightweight and memory-efficient fine-tuning method -- to overcome the limitations of the current PETL approaches. Specifically, READ inserts a small RNN network alongside the backbone model so that the model does not have to back-propagate through the large backbone network. Through comprehensive empirical evaluation of the GLUE benchmark, we demonstrate READ can achieve a $56\%$ reduction in the training memory consumption and an $84\%$ reduction in the GPU energy usage while retraining high model quality compared to full-tuning. Additionally, the model size of READ does not grow with the backbone model size, making it a highly scalable solution for fine-tuning large Transformers.

Motivation & Objective

• Address the high computational and memory costs of fine-tuning large-scale Transformers, especially as model size and task count grow.
• Overcome the limitations of existing parameter-efficient transfer learning (PETL) methods that still require significant energy and memory due to backpropagation through the large backbone.
• Eliminate the need for pretraining the side network, a prerequisite in prior side-tuning methods like Ladder-Side Tuning.
• Develop a scalable, lightweight solution that maintains high model quality while drastically reducing training resource consumption.
• Enable practical deployment of fine-tuned large models in resource-constrained environments by minimizing both memory and energy usage.

Proposed method

• Insert a small, trainable RNN network (READ) as a side network alongside the frozen Transformer backbone, operating on cached intermediate activations.
• Use a Joiner network to combine multiple information sources—such as input embeddings and intermediate hidden states—into the RNN input at each layer.
• Perform forward passes independently on the backbone and READ network, caching intermediate representations to avoid recomputation.
• Compute RNN hidden states iteratively across encoder and decoder layers, then add the RNN output to the backbone's output to produce the final prediction.
• Train only the RNN parameters and the final classification head, keeping all backbone weights frozen to reduce parameter updates and backpropagation cost.
• Eliminate the need for pretraining the side network, unlike Ladder-Side Tuning, by directly initializing and training the RNN from scratch.

Figure 1: The normalized energy consumption relative to full-tuning on GLUE tasks.

Experimental results

Research questions

• RQ1 Can a lightweight RNN-based side network achieve competitive performance with full fine-tuning while drastically reducing memory and energy consumption?
• RQ2 Does eliminating the need for pretraining the side network in side-tuning methods improve efficiency and scalability?
• RQ3 How does READ compare to existing PETL methods (e.g., LoRA, adapters, prompt tuning) in terms of memory, energy, and inference latency?
• RQ4 Can READ maintain high model quality across diverse NLP benchmarks like GLUE without increasing model size or inference overhead?
• RQ5 Is READ's performance and efficiency scalable across different backbone sizes, including large and 3B-parameter T5 models?

Key findings

• READ reduces peak training memory consumption by 56% compared to full fine-tuning on the GLUE benchmark.
• READ achieves an 84% reduction in GPU energy usage relative to full fine-tuning, significantly lowering training costs.
• The method maintains competitive model accuracy across all 7 GLUE tasks, matching or exceeding the performance of full fine-tuning and other PETL methods.
• READ introduces no additional inference latency or memory overhead, as the RNN is only active during training.
• The size of the READ network does not scale with the backbone model size, making it highly scalable for large-scale Transformers.
• READ outperforms Ladder-Side Tuning (LST) in both efficiency and simplicity, avoiding the need for pretraining the side network and reducing training complexity.

Figure 2: Comparing READ and other fine-tuning methods over GLUE tasks on training energy and peak training memory relative to full-tuning. The y-axis is the average metric over 7 GLUE tasks. (left) The x-axis is the cumulative training energy in KWh. (right) The x-axis is the GPU peak training memo