[Paper Review] Parameter and Computation Efficient Transfer Learning for Vision-Language Pre-trained Models

Qiong Wu, Wei Yu+4 more|arXiv (Cornell University)|Sep 4, 2023

Multimodal Machine Learning Applications4 citations

TL;DR

This paper proposes Dynamic Architecture Skipping (DAS), a parameter and computation efficient transfer learning (PCETL) method for vision-language pre-trained (VLP) models. DAS uses reinforcement learning to identify and skip redundant Transformer modules in VLP models during inference, while inserting lightweight adapters for feature adaptation, achieving up to 19.23% faster inference and 11.97% fewer FLOPs on VQA2.0 without sacrificing performance.

ABSTRACT

With ever increasing parameters and computation, vision-language pre-trained (VLP) models exhibit prohibitive expenditure in downstream task adaption. Recent endeavors mainly focus on parameter efficient transfer learning (PETL) for VLP models by only updating a small number of parameters. However, excessive computational overhead still plagues the application of VLPs. In this paper, we aim at parameter and computation efficient transfer learning (PCETL) for VLP models. In particular, PCETL not only needs to limit the number of trainable parameters in VLP models, but also to reduce the computational redundancy during inference, thus enabling a more efficient transfer. To approach this target, we propose a novel dynamic architecture skipping (DAS) approach towards effective PCETL. Instead of directly optimizing the intrinsic architectures of VLP models, DAS first observes the significances of their modules to downstream tasks via a reinforcement learning (RL) based process, and then skips the redundant ones with lightweight networks, i.e., adapters, according to the obtained rewards. In this case, the VLP model can well maintain the scale of trainable parameters while speeding up its inference on downstream tasks. To validate DAS, we apply it to two representative VLP models, namely ViLT and METER, and conduct extensive experiments on a bunch of VL tasks. The experimental results not only show the great advantages of DAS in reducing computational complexity, e.g. -11.97% FLOPs of METER on VQA2.0, but also confirm its competitiveness against existing PETL methods in terms of parameter scale and performance. Our source code is given in our appendix.

Motivation & Objective

• To address the high computational cost of fine-tuning large vision-language pre-trained (VLP) models on downstream tasks despite existing parameter-efficient transfer learning (PETL) methods.
• To reduce both parameter and computation overhead during inference by identifying and skipping redundant modules in pre-trained VLP models.
• To maintain competitive performance while significantly accelerating inference through dynamic subnetwork routing based on task-specific module importance.
• To enable efficient transfer learning that is both parameter-efficient and computation-efficient, overcoming limitations of current PETL approaches.

Proposed method

• DAS frames module skipping as a k-armed bandit problem, using reinforcement learning (RL) to evaluate the importance of each vision-language (VL) module for a downstream task.
• The RL agent selects which Transformer layers to skip based on cumulative rewards, dynamically routing the model to an optimal subnetwork for inference.
• Redundant modules are replaced with lightweight adapters and skip connections, preserving feature adaptation capability while reducing computation.
• The method maintains a fixed number of trainable parameters comparable to PETL methods, ensuring parameter efficiency.
• DAS operates post-pretraining, avoiding retraining or architectural redesign, making it applicable to already well-pretrained VLP models.
• The search process is guided by a reward function based on module redundancy estimation, with results visualized to confirm task-specific routing patterns.

Experimental results

Research questions

• RQ1 Can a dynamic architecture skipping mechanism reduce computational redundancy in vision-language pre-trained models during inference?
• RQ2 How does the performance of DAS compare to existing PETL methods in terms of parameter efficiency and inference speed?
• RQ3 To what extent can DAS reduce FLOPs and accelerate inference while maintaining competitive accuracy on vision-language tasks?
• RQ4 What are the task-specific patterns in module skipping, and how do they reflect the semantic demands of different VL tasks?

Key findings

• DAS reduces FLOPs by 11.97% on METER for the VQA2.0 task, demonstrating significant computational savings.
• DAS improves inference speed by 19.23% compared to full fine-tuning, outperforming even standard PETL methods in inference efficiency.
• On three vision-language tasks, DAS maintains competitive performance while reducing computational overhead, proving its effectiveness in PCETL.
• The RL-based module selection process successfully identifies redundant layers early in training, with redundancy estimates stabilizing after oscillation.
• Task-specific subnetworks are discovered: VQA favors skipping middle-level language modules, while NLVR prefers skipping top-level language layers, reflecting task semantics.
• DAS achieves a favorable trade-off between parameter efficiency and computation efficiency, being the only PETL method that improves inference speed over the full model.