[Paper Review] TRACE: A Comprehensive Benchmark for Continual Learning in Large Language Models

Xiao Wang, Yuansen Zhang+10 more|arXiv (Cornell University)|Oct 10, 2023

Topic Modeling4 citations

TL;DR

TRACE introduces a comprehensive benchmark for evaluating continual learning in aligned large language models (LLMs), comprising 8 diverse, standardized datasets across domain-specific tasks, multilingual capabilities, code generation, and mathematical reasoning. The study reveals severe catastrophic forgetting—e.g., Llama2-13B's GSM8K accuracy dropped from 28.8% to 2%—and proposes Reasoning-augmented Continual Learning (RCL), which uses meta-rationales and task-specific cues to reduce forgetting and accelerate convergence.

ABSTRACT

Aligned large language models (LLMs) demonstrate exceptional capabilities in task-solving, following instructions, and ensuring safety. However, the continual learning aspect of these aligned LLMs has been largely overlooked. Existing continual learning benchmarks lack sufficient challenge for leading aligned LLMs, owing to both their simplicity and the models' potential exposure during instruction tuning. In this paper, we introduce TRACE, a novel benchmark designed to evaluate continual learning in LLMs. TRACE consists of 8 distinct datasets spanning challenging tasks including domain-specific tasks, multilingual capabilities, code generation, and mathematical reasoning. All datasets are standardized into a unified format, allowing for effortless automatic evaluation of LLMs. Our experiments show that after training on TRACE, aligned LLMs exhibit significant declines in both general ability and instruction-following capabilities. For example, the accuracy of llama2-chat 13B on gsm8k dataset declined precipitously from 28.8\% to 2\% after training on our datasets. This highlights the challenge of finding a suitable tradeoff between achieving performance on specific tasks while preserving the original prowess of LLMs. Empirical findings suggest that tasks inherently equipped with reasoning paths contribute significantly to preserving certain capabilities of LLMs against potential declines. Motivated by this, we introduce the Reasoning-augmented Continual Learning (RCL) approach. RCL integrates task-specific cues with meta-rationales, effectively reducing catastrophic forgetting in LLMs while expediting convergence on novel tasks.

Motivation & Objective

• To address the lack of challenging, realistic benchmarks for evaluating continual learning in aligned LLMs.
• To investigate catastrophic forgetting in state-of-the-art LLMs when trained incrementally on diverse, complex tasks.
• To evaluate the impact of continual learning on general abilities, instruction following, and safety in LLMs.
• To explore whether reasoning-augmented training can mitigate forgetting while accelerating learning on new tasks.
• To provide a standardized, evaluable benchmark with unified formats and novel metrics for continual learning evaluation.

Proposed method

• Designing TRACE, a benchmark with 8 diverse, standardized datasets covering domain-specific tasks, multilingual understanding, code generation, and mathematical reasoning.
• Sampling 5,000 balanced instances per task with class-balanced splits for classification tasks.
• Introducing three novel evaluation metrics: General Ability Delta, Instruction Following Delta, and Safety Delta to quantify forgetting across core LLM capabilities.
• Using GPT-4 as a judge for automated, objective evaluation of model responses in safety and helpfulness assessments.
• Proposing Reasoning-augmented Continual Learning (RCL), which prompts models to generate rationales and task analyses during training.
• Applying RCL with meta-rationales and task-specific cues to improve retention of original model capabilities during incremental training.

Experimental results

Research questions

• RQ1 To what extent do aligned LLMs suffer from catastrophic forgetting when trained incrementally on diverse, complex tasks?
• RQ2 How does continual learning affect general reasoning, instruction following, and safety capabilities in LLMs?
• RQ3 Can reasoning-augmented training mitigate catastrophic forgetting while improving convergence on new tasks?
• RQ4 What role do reasoning paths play in preserving model capabilities during continual learning?
• RQ5 How do full-parameter fine-tuning and parameter-efficient fine-tuning (LoRA) compare in terms of forgetting and performance?

Key findings

• Llama2-13B's accuracy on GSM8K dropped from 28.8% to 2% after continual training on TRACE, indicating severe catastrophic forgetting in mathematical reasoning.
• Multilingual capabilities improved during continual learning, with Llama2-7B’s TydiQA F1 score rising from 23.47 to 33.23, suggesting positive transfer in this domain.
• Full-parameter fine-tuning led to stronger task fitting but significantly worse forgetting in general abilities compared to LoRA.
• Instruction-following capabilities declined substantially after continual learning, highlighting a critical vulnerability in aligned LLMs.
• Tasks with explicit reasoning paths preserved model capabilities better than those without, suggesting reasoning integration mitigates forgetting.
• The proposed RCL method reduced forgetting and accelerated convergence by prompting models to generate rationales and task analyses during training.