[Paper Review] Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges

Nianwen Si, Hao Zhang+4 more|arXiv (Cornell University)|Nov 27, 2023

Topic Modeling4 citations

TL;DR

This survey introduces knowledge unlearning for large language models (LLMs), proposing a framework to efficiently remove harmful or incorrect knowledge without retraining the entire model. It categorizes methods into parameter optimization, parameter merging, and in-context learning, demonstrating that targeted unlearning preserves unrelated knowledge while mitigating risks of malicious or faulty outputs.

ABSTRACT

In recent years, large language models (LLMs) have spurred a new research paradigm in natural language processing. Despite their excellent capability in knowledge-based question answering and reasoning, their potential to retain faulty or even harmful knowledge poses risks of malicious application. The challenge of mitigating this issue and transforming these models into purer assistants is crucial for their widespread applicability. Unfortunately, Retraining LLMs repeatedly to eliminate undesirable knowledge is impractical due to their immense parameters. Knowledge unlearning, derived from analogous studies on machine unlearning, presents a promising avenue to address this concern and is notably advantageous in the context of LLMs. It allows for the removal of harmful knowledge in an efficient manner, without affecting unrelated knowledge in the model. To this end, we provide a survey of knowledge unlearning in the era of LLMs. Firstly, we formally define the knowledge unlearning problem and distinguish it from related works. Subsequently, we categorize existing knowledge unlearning methods into three classes: those based on parameter optimization, parameter merging, and in-context learning, and introduce details of these unlearning methods. We further present evaluation datasets used in existing methods, and finally conclude this survey by presenting the ongoing challenges and future directions.

Motivation & Objective

• Address the risk of LLMs retaining harmful or incorrect knowledge that could lead to malicious or biased outputs.
• Overcome the impracticality of full retraining for knowledge deletion in large-scale LLMs.
• Systematically categorize and analyze existing knowledge unlearning methods in the context of LLMs.
• Provide evaluation benchmarks and identify open challenges for future research in the field.

Proposed method

• Formally define the knowledge unlearning problem, distinguishing it from related concepts like data extraction and model distillation.
• Classify unlearning methods into three categories: parameter optimization (e.g., fine-tuning with unlearning signals), parameter merging (e.g., combining model weights with unlearning updates), and in-context learning (e.g., prompt-based unlearning without weight modification).
• Propose a taxonomy of unlearning techniques based on their mechanism of knowledge removal and impact on model parameters.
• Introduce evaluation datasets used to benchmark unlearning performance across different methods.
• Analyze the trade-offs between unlearning effectiveness, model accuracy on unrelated tasks, and computational efficiency.
• Highlight the importance of preserving factual knowledge not related to the unlearned data, ensuring model utility post-unlearning.

Experimental results

Research questions

• RQ1 What defines knowledge unlearning in the context of LLMs, and how does it differ from related tasks like model distillation or data extraction?
• RQ2 How effective are parameter optimization, parameter merging, and in-context learning methods in removing specific harmful or incorrect knowledge from LLMs?
• RQ3 To what extent can unlearning be achieved without degrading the model’s performance on unrelated, non-targeted knowledge?
• RQ4 What evaluation benchmarks and metrics are currently used to assess unlearning effectiveness in LLMs?
• RQ5 What are the key challenges and open problems in scaling and generalizing knowledge unlearning methods across diverse LLM architectures and datasets?

Key findings

• Knowledge unlearning is feasible in LLMs through targeted methods that avoid full retraining, significantly reducing computational cost.
• Parameter optimization methods show strong unlearning performance but risk overfitting to unlearned examples if not carefully regularized.
• Parameter merging techniques preserve more general knowledge and exhibit better robustness compared to fine-tuning-based approaches.
• In-context learning offers a parameter-free unlearning solution, though its effectiveness is limited to specific prompt designs and may not generalize across diverse knowledge types.
• Evaluation benchmarks for knowledge unlearning are still emerging, with limited standardization across datasets and metrics.
• A critical challenge remains balancing unlearning effectiveness with the preservation of unrelated factual knowledge, especially in high-parameter models.