[Paper Review] FeDeRA:Efficient Fine-tuning of Language Models in Federated Learning Leveraging Weight Decomposition

Yuxuan Yan, Qian‐Qian Yang+2 more|arXiv (Cornell University)|Apr 29, 2024

Privacy-Preserving Technologies in Data4 citations

TL;DR

FeDeRA proposes a novel federated fine-tuning method for large language models that leverages Singular Value Decomposition (SVD) to initialize LoRA adapter modules, significantly improving performance under non-IID data distributions. It achieves near-full fine-tuning accuracy while reducing training time by up to 97.9% compared to full-parameter fine-tuning on edge devices.

ABSTRACT

Despite their exceptional performance on various tasks after fine-tuning, pre-trained language models (PLMs) face significant challenges due to growing privacy concerns with data in centralized training methods. We consider federated learning (FL) to fine-tune PLMs in this paper. However, the substantial number of parameters in PLMs poses significant difficulties for client devices with limited communication and computational resources. One promising solution is to exploit parameter-efficient fine-tuning (PEFT) into FL, which trains a much smaller set of parameters than full parameter fine-tuning (FFT). Although remarkably improving training efficiency, PEFT methods may lead to degraded performance especially when data across different clients are non i.i.d, as revealed by experimental results. To overcome this, we propose FeDeRA, which extends and improves a widely used PEFT method, i.e., low-rank adaption (LoRA). FeDeRA follows LoRA by decomposing the weight matrices of the PLMs into low-rank matrices, which allows for more efficient computation and parameter updates during fine-tuning. Different from LoRA which simply initializes these low-rank matrices by random sampling or zeros, the proposed FeDeRA initializes these matrices by the results of performing singular value decomposition (SVD) on the pre-trained weight matrices. Extensive experiments across various tasks and datasets show that FeDeRA outperforms the considered PEFT baselines and is comparable to or even surpasses FFT method within the FL setting in terms of task performance. Moreover, FeDeRA requires only 1% trainable paramentes compared to FFT, significantly reducing training time costs by more than 90% to achieve the same task performance level. The experimental results also highlight the robustness of FeDeRA against data heterogeneity, as it maintains stable task performance even as data heterogeneity increases.

Motivation & Objective

• To address the performance degradation of parameter-efficient fine-tuning (PEFT) methods like LoRA in federated learning due to non-IID data distributions.
• To reduce communication and computational overhead in federated fine-tuning of large pre-trained language models (PLMs).
• To maintain high model accuracy comparable to full-parameter fine-tuning (FFT) while minimizing trainable parameters and training time.
• To enable efficient, privacy-preserving fine-tuning on resource-constrained edge devices such as Jetson AGX Orin.

Proposed method

• FeDeRA initializes LoRA adapter modules by performing Singular Value Decomposition (SVD) on the pre-trained weight matrices and selecting the principal components as the initial adapter weights.
• The method retains and freezes the original pre-trained weights, only updating the SVD-derived adapter components during fine-tuning.
• The adapter module structure remains identical to LoRA, ensuring low computational and memory overhead.
• SVD is applied only once at initialization, making the additional cost negligible compared to full training.
• The approach is integrated into a federated learning framework where clients train locally and send only adapter parameters to the server for aggregation.
• The method is evaluated on RoBERTa-base and DeBERTaV3-base across three NLP tasks with varying data heterogeneity levels.

Experimental results

Research questions

• RQ1 Can SVD-based initialization of LoRA adapters mitigate performance degradation in federated fine-tuning under non-IID data distributions?
• RQ2 How does FeDeRA compare to full-parameter fine-tuning (FFT) in terms of model accuracy and training efficiency in federated settings?
• RQ3 To what extent does FeDeRA reduce communication and training time compared to FFT and other PEFT methods on edge devices?
• RQ4 Does the SVD initialization improve convergence speed and final performance in federated learning with heterogeneous data?

Key findings

• FeDeRA outperforms all other PEFT methods and achieves performance comparable to or better than full-parameter fine-tuning (FFT) across three tasks and six datasets.
• On RoBERTa-base, FeDeRA reduces training time by 95.9%, 97.9%, and 96.9% compared to FFT to reach 99% of the target accuracy on three datasets.
• On DeBERTaV3-base, FeDeRA reduces training time by 97.3%, 96.5%, and 96.5% compared to FFT under the same accuracy targets.
• The SVD initialization step adds only a negligible one-time cost, as it is performed once at model initialization and not during training.
• In real-world federated experiments using Jetson AGX Orin devices, FeDeRA achieved the shortest training time across all methods while maintaining high accuracy.
• FeDeRA effectively mitigates the performance gap caused by data heterogeneity, outperforming standard LoRA and other PEFT methods in non-IID settings.