[Paper Review] BiLoRA: A Bi-level Optimization Framework for Overfitting-Resilient Low-Rank Adaptation of Large Pre-trained Models

Rushi Qiang, Ruiyi Zhang+1 more|arXiv (Cornell University)|Mar 19, 2024

Advanced Neural Network Applications4 citations

TL;DR

BiLoRA proposes a bi-level optimization framework to mitigate overfitting in Low-Rank Adaptation (LoRA) for fine-tuning large pre-trained models. By splitting the training of pseudo singular vectors and values across disjoint data subsets using SVD parameterization, BiLoRA improves generalization and reduces training time, outperforming LoRA and AdaLoRA across 10 NLP tasks with similar parameter counts.

ABSTRACT

Low-rank adaptation (LoRA) is a popular method for fine-tuning large-scale pre-trained models in downstream tasks by learning low-rank incremental matrices. Though LoRA and its variants effectively reduce the number of trainable parameters compared to full fine-tuning methods, they often overfit training data, resulting in sub-optimal generalization on test data. To address this problem, we introduce BiLoRA, an overfitting-alleviating fine-tuning approach based on bi-level optimization (BLO). BiLoRA employs pseudo singular value decomposition to parameterize low-rank incremental matrices and splits the training of pseudo singular vectors and values across two different subsets of training data. This division, embedded within separate levels of the BLO framework, mitigates the risk of overfitting to a single dataset. Tested on ten datasets covering natural language understanding and generation tasks and applied to various well-known large pre-trained models, BiLoRA significantly outperforms LoRA methods and other fine-tuning approaches, with similar amounts of trainable parameters.

Motivation & Objective

• Address the persistent overfitting issue in LoRA and its variants during fine-tuning of large pre-trained models.
• Improve model generalization on test data without increasing the number of trainable parameters.
• Reduce training time compared to standard LoRA while maintaining or improving performance.
• Develop a parameter-efficient fine-tuning method that leverages bi-level optimization to decouple the learning of singular vectors and values.
• Provide a robust, hyperparameter-insensitive method that avoids the need for extensive tuning of regularization weights.

Proposed method

• Parameterize low-rank update matrices using pseudo singular value decomposition (SVD) as ΔW = PΛQ, where P and Q are pseudo singular vectors and Λ is the pseudo singular value matrix.
• Apply orthogonality-promoting regularization to ensure P and Q remain approximately orthogonal during training.
• Split the training data into two disjoint subsets: one for training P and Q at the lower optimization level, and another for validating Λ at the upper level.
• Train P and Q on a subset S by minimizing fine-tuning loss while holding Λ constant, producing P*(Λ) and Q*(Λ).
• Optimize Λ at the upper level by minimizing the validation loss on D\S, using the optimal P*(Λ) and Q*(Λ) as inputs.
• Use a softplus parameterization for Λ to allow differentiable and stable optimization, enabling larger learning rates and faster convergence.

Experimental results

Research questions

• RQ1 Can bi-level optimization effectively reduce overfitting in LoRA fine-tuning without increasing the number of trainable parameters?
• RQ2 How does separating the training of singular vectors and values across disjoint data subsets improve model generalization?
• RQ3 Does the proposed method reduce training time compared to standard LoRA and AdaLoRA while maintaining or improving performance?
• RQ4 How robust is BiLoRA to hyperparameter choices, particularly the regularization weight for orthogonality?
• RQ5 Can the bi-level framework be effectively applied to diverse NLP tasks, including both natural language understanding and generation?

Key findings

• BiLoRA significantly outperforms LoRA and AdaLoRA across 10 diverse NLP datasets, including CoLA, SST-2, MNLI, and QQP, with similar numbers of trainable parameters.
• On the RoBERTa-base model, BiLoRA reduced total training time from 7,802.1 minutes (LoRA) to 4,531.4 minutes, a 42.5% reduction.
• On RoBERTa-large, BiLoRA reduced training time from 2,661.3 minutes (LoRA) to 2,363.8 minutes, a 11.2% reduction.
• The method is robust to hyperparameter choices: performance remains stable across different values of the orthogonality regularization weight γ₁, reducing the need for extensive tuning.
• Increasing weight decay in AdaLoRA worsened performance, indicating that traditional regularization fails to address overfitting in LoRA, validating the need for BiLoRA’s novel approach.
• BiLoRA converges in fewer training epochs than LoRA, and despite higher per-step cost due to bi-level computation, the total training time is lower due to faster convergence.