[论文解读] LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition

Chengsong Huang, Qian Liu+4 more|arXiv (Cornell University)|Jul 25, 2023

Context-Aware Activity Recognition Systems被引用 7

一句话总结

LoraHub 自动组合预训练的 LoRA 模块以使用少量示例将大语言模型适配到未见任务，达到接近上下文学习的性能，同时输入令牌更少且无额外参数。

ABSTRACT

Low-rank adaptations (LoRA) are often employed to fine-tune large language models (LLMs) for new tasks. This paper investigates LoRA composability for cross-task generalization and introduces LoraHub, a simple framework devised for the purposive assembly of LoRA modules trained on diverse given tasks, with the objective of achieving adaptable performance on unseen tasks. With just a few examples from a new task, LoraHub can fluidly combine multiple LoRA modules, eliminating the need for human expertise and assumptions. Notably, the composition requires neither additional model parameters nor gradients. Empirical results on the Big-Bench Hard benchmark suggest that LoraHub, while not surpassing the performance of in-context learning, offers a notable performance-efficiency trade-off in few-shot scenarios by employing a significantly reduced number of tokens per example during inference. Notably, LoraHub establishes a better upper bound compared to in-context learning when paired with different demonstration examples, demonstrating its potential for future development. Our vision is to establish a platform for LoRA modules, empowering users to share their trained LoRA modules. This collaborative approach facilitates the seamless application of LoRA modules to novel tasks, contributing to an adaptive ecosystem. Our code is available at https://github.com/sail-sg/lorahub, and all the pre-trained LoRA modules are released at https://huggingface.co/lorahub.

研究动机与目标

• 使用在多样化任务上训练的模块化 LoRA 调整器研究跨任务泛化能力。
• 开发一个两阶段框架（组合Compose与适应Adapt），为未见任务组装并调优 LoRA 模块。
• 在适应过程中消除手动模块选择和额外的模型参数。
• 在 BBH 与 FLAN-T5 的对比中评估与零-shot 与上下文学习的效率与性能权衡。
• 提出面向平台的分享 LoRA 模块的愿景，促进可重用组件的协作。

提出的方法

• 在不同上游任务上训练多种 LoRA 模块以捕捉多样化技能。
• 组合阶段：用标量系数 w_i 将 LoRA 模块线性组合成单一 hat{m}。
• 適应阶段：对少量示例应用无梯度优化（如 CMA-ES）来调整 w，使目标函数 L + alpha * sum|w_i| 最小化。
• 定稿：将优化后的 hat{m} 与基础 LLM 集成，以执行未见任务且不再更新参数。
• 通过保持输入简洁并在推理阶段避免逐例演示来评估推理效率。

Figure 1: The illustration of zero-shot learning, few-shot in-context learning and few-shot LoraHub learning (ours). Note that the Compose procedure is conducted per task rather than per example. Our method achieves similar inference throughput as zero-shot learning, yet approaches the performance o

实验结果

研究问题

• RQ1 diverse Task 上训练的 LoRA 模块库是否能有效组合以对未见任务实现泛化？
• RQ2 对模块系数进行无梯度优化是否能在少量示例下实现有竞争力的性能？
• RQ3 与零样本和上下文学习相比，LoraHub 在标准基准上的效率与性能权衡为何？
• RQ4 LoRA 模块的数量与秩对跨任务泛化与稳定性有何影响？

主要发现

• LoraHub 在少量示例的 BBH 设置中接近上下文学习的性能，同时对每个示例使用相同或更少的令牌。
• 当搭配不同的演示示例时，该方法可以实现对 ICL 的显著上限优势。
• 无梯度 CMA-ES 优化在选择合适的 LoRA 权重方面具有有效性，所需计算量适中（如在单个 A100 上不到一分钟）。
• LoraHub 可以在仅 CPU 处理 LoRA 的情况下运行，前提是提供了 LLM 推理，从而实现更大众化的工作流。
• 与基于梯度的微调相比，LoraHub 保留了基础模型参数，且推理阶段避免逐例提示。

Figure 2: Our method encompasses two stages: the Compose stage and the Adapt stage. During the Compose stage, existing LoRA modules are integrated into one unified module, employing a set of coefficients, denoted as $w$ . In the Adapt stage, the combined LoRA module is evaluated on a few examples fr