Skip to main content
QUICK REVIEW

[论文解读] GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Jiawei Zhao, Zhenyu Zhang|arXiv (Cornell University)|Mar 6, 2024
Neural Networks and ApplicationsComputer Science被引用 10
一句话总结

GaLore 使用内存高效的梯度低秩投影训练全参数模型,在保持 LLM 预训练与微调性能的同时,显著降低优化器状态的内存占用。

ABSTRACT

Training Large Language Models (LLMs) presents significant memory challenges, predominantly due to the growing size of weights and optimizer states. Common memory-reduction approaches, such as low-rank adaptation (LoRA), add a trainable low-rank matrix to the frozen pre-trained weight in each layer, reducing trainable parameters and optimizer states. However, such approaches typically underperform training with full-rank weights in both pre-training and fine-tuning stages since they limit the parameter search to a low-rank subspace and alter the training dynamics, and further, may require full-rank warm start. In this work, we propose Gradient Low-Rank Projection (GaLore), a training strategy that allows full-parameter learning but is more memory-efficient than common low-rank adaptation methods such as LoRA. Our approach reduces memory usage by up to 65.5% in optimizer states while maintaining both efficiency and performance for pre-training on LLaMA 1B and 7B architectures with C4 dataset with up to 19.7B tokens, and on fine-tuning RoBERTa on GLUE tasks. Our 8-bit GaLore further reduces optimizer memory by up to 82.5% and total training memory by 63.3%, compared to a BF16 baseline. Notably, we demonstrate, for the first time, the feasibility of pre-training a 7B model on consumer GPUs with 24GB memory (e.g., NVIDIA RTX 4090) without model parallel, checkpointing, or offloading strategies.

研究动机与目标

  • 阐明大型语言模型在预训练与微调过程中的内存瓶颈。
  • 提出基于梯度的低秩投影,以降低优化器内存占用,同时不限制全参数学习。
  • 在 LLaMA-7B 的预训练和 RoBERTa-GLE 的微调上展示内存与性能收益。
  • 展示与现有优化器及内存节省技术的兼容性。

提出的方法

  • 利用权重梯度在训练过程中趋于低秩的观测(引理 3.1 及相关推论)。
  • 引入两个投影矩阵 P ∈ R^{m×r} 和 Q ∈ R^{n×r},构成 P^T G Q,用于梯度更新。
  • 通过 tilde{G}_t = P_t ρ_t(P_t^T G_t Q_t) Q_t^T 来更新权重,保持全参数学习。
  • 通过在预定间隔(T)使用基于 SVD 的投影重新初始化 P 和 Q,允许子空间切换。
  • 给出收敛性分析(定理 3.6),在固定或期日更新的投影下证明 GaLore 收敛。
  • 展示与 Adam、8-bit Adam、Adafactor 以及逐层权重更新的兼容性。
  • 将 GaLore 与内存高效优化器结合,以进一步降低内存开销。
Figure 1 : Memory consumption of pre-training a LLaMA 7B model with a token batch size of 256 on a single device, without activation checkpointing and memory offloading. Details refer to Section 5.5 .
Figure 1 : Memory consumption of pre-training a LLaMA 7B model with a token batch size of 256 on a single device, without activation checkpointing and memory offloading. Details refer to Section 5.5 .

实验结果

研究问题

  • RQ1在大型语言模型训练中,梯度矩阵能否有效地被视为低秩以在不影响性能的前提下降低内存占用?
  • RQ2在实现与 LoRA 基于方法相当或更优的内存节省的同时,GaLore 是否能够维持全参数学习?
  • RQ3子空间切换的频率与秩对收敛性和最终模型质量有何影响?
  • RQ4GaLore 是否兼容常见的内存高效优化器和训练方案(预训练和微调)?

主要发现

  • GaLore 在使用 8 位优化器时实现了优化器内存最多 65.5% 的减少,以及总训练内存最多 63.3% 的减少(相较 BF16 基线)。
  • 在对 C4 进行 19.7B 标记的 LLaMA 7B 预训练中,8-bit GaLore 的困惑度与全秩训练相当,但内存显著减少;在某些设置下,在较高 r 时与或略超越全秩性能。
  • GaLore 使在消费级GPU(如 RTX 4090,24GB 内存)上对 7B 模型进行预训练成为可能,无需模型并行、检查点或 offloading(使用激活检查点时可以进一步扩展批量大小)。
  • 在 GLUE 上进行 RoBERTa-Base 微调时,GaLore(秩为 4)在平均 GLUE 分数上优于 LoRA(85.89 对 85.61),且内存占用相近或更小;秩为 8 的 GaLore 也带来可观提升。
  • GaLore 仍然与优化器无关,适用于 AdamW、8-bit Adam 和 Adafactor,并支持逐层权重更新以实现额外的内存节省。
Figure 2 : Learning through low-rank subspaces $\Delta W_{T_{1}}$ and $\Delta W_{T_{2}}$ using GaLore. For $t_{1}\in[0,T_{1}-1]$ , $W$ are updated by projected gradients $\tilde{G}_{t_{1}}$ in a subspace determined by fixed $P_{t_{1}}$ and $Q_{t_{1}}$ . After $T_{1}$ steps, the subspace is changed b
Figure 2 : Learning through low-rank subspaces $\Delta W_{T_{1}}$ and $\Delta W_{T_{2}}$ using GaLore. For $t_{1}\in[0,T_{1}-1]$ , $W$ are updated by projected gradients $\tilde{G}_{t_{1}}$ in a subspace determined by fixed $P_{t_{1}}$ and $Q_{t_{1}}$ . After $T_{1}$ steps, the subspace is changed b

阅读本论文

在 Nubint 阅读器中打开原文,边读边就任意段落提问。

在 Nubint 中阅读本论文

本解读由 AI 根据论文原文生成,引用前请核对原文。