[Paper Review] Fine Tuning LLM for Enterprise: Practical Guidelines and Recommendations

Mathav Raj J, Kushala VM+2 more|arXiv (Cornell University)|Mar 23, 2024

Metallurgy and Material Forming30 citations

TL;DR

The paper provides practical guidance for fine-tuning LLaMA on proprietary documents and code using PEFT methods (LoRA/QLoRA), quantization, and custom data preprocessing, with empirical evaluation and RAG comparison.

ABSTRACT

There is a compelling necessity from enterprises for fine tuning LLMs (Large Language Models) o get them trained on proprietary domain knowledge. The challenge is to imbibe the LLMs with domain specific knowledge using the most optimial resource and cost and in the best possible time. Many enterprises rely on RAG (Retrieval Augmented Generation) which does not need LLMs to be ine-tuned but they are limited by the quality of vector databases and their retrieval capabilities rather than the intrinsic capabilities of the LLMs themselves. In our current work we focus on fine tuning LLaMA, an open source LLM using proprietary documents and code from an enterprise repository and use the fine tuned models to evaluate the quality of responses. As part of this work, we aim to guide beginners on how to start with fine tuning an LLM for documentation and code by making educated guesses on size of GPU required and options that are available for formatting the data. We also propose pre processing recipes for both documentation and code to prepare dataset in different formats. The proposed methods of data preparation for document datasets are forming paragraph chunks, forming question and answer pairs and forming keyword and paragraph chunk pairs. For code dataset we propose forming summary and function pairs. Further, we qualitatively evaluate the results of the models for domain specific queries. Finally, we also propose practical guidelines and recommendations for fine tuning LLMs.

Motivation & Objective

• Motivate enterprises to fine-tune LLMs on domain-specific data to improve accuracy and reduce latency and privacy concerns.
• Guide beginners on data preparation, compute estimates, and dataset formats for text and code fine-tuning.
• Evaluate the impact of quantization, gradient accumulation, and PEFT (LoRA/QLoRA) on resource use and performance.
• Provide actionable recommendations and workflows for fine-tuning LLMs in enterprise settings.

Proposed method

• Discuss factors influencing fine-tuning on available hardware, including memory and compute trade-offs.
• Describe data preprocessing recipes for text and code datasets to create instruction-tuned formats.
• Explain PEFT techniques (LoRA and QLoRA) and their parameter efficiency benefits for different model sizes.
• Present workflow: data preprocessing, PEFT setup, fine-tuning until loss minimization, and evaluation in RAG pipelines.
• Conduct empirical experiments using LLaMA 2 on proprietary document and code corpora to study quantization, LoRA rank/alpha, and full fine-tuning feasibility.

Figure 1: Fine tuning Workflow (with LLaMA model as an example)

Experimental results

Research questions

• RQ1 How do quantization, PEFT methods (LoRA/QLoRA), and gradient accumulation affect fine-tuning efficiency and memory usage on enterprise-scale LLMs?
• RQ2 What data preprocessing formats for text and code yield the best domain-specific performance in instruction-tuned LLaMA models?
• RQ3 How does fine-tuning influence RAG-based retrieval quality and tendency to hallucinate on enterprise documentation and code?
• RQ4 What are practical guidelines for selecting model size, dataset size, and training configurations given fixed hardware resources?

Key findings

• Quantization (e.g., 8-bit) reduces GPU memory by about 71% and enables larger batch sizes, at the cost of longer inference time during fine-tuning.
• LoRA/QLoRA enable parameter-efficient fine-tuning, allowing fine-tuning of large models on limited memory, with full model fine-tuning still feasible for smaller sizes on high-end GPUs.
• Data preprocessing choices (raw, keyword, heading, and summary-based formats) produce different response qualities, with some formats yielding more concise or detailed answers depending on the task.
• In RAG pipelines, fine-tuned models provide more accurate and document-styled, stepwise responses, reducing hallucinations compared to base pretrained models.
• Code-focused fine-tuning shows strong gains with appropriately tuned LoRA rank and alpha, while excessive rank/alpha can lead to hallucinations in generated code.

Figure 2: Inference time of fine tuned Llama 2 7B Chat model