[Paper Review] TOFU: A Task of Fictitious Unlearning for LLMs

Pratyush Maini, Zhili Feng+3 more|arXiv (Cornell University)|Jan 11, 2024

Topic Modeling4 citations

TL;DR

This paper introduces TOFU, a benchmark for evaluating unlearning in large language models (LLMs) using synthetic author profiles to test whether models can effectively forget specific data. Despite testing four baseline unlearning methods, the results show that existing approaches fail to achieve effective unlearning, highlighting a critical gap in current techniques for privacy-preserving LLM tuning.

ABSTRACT

Large language models trained on massive corpora of data from the web can memorize and reproduce sensitive or private data raising both legal and ethical concerns. Unlearning, or tuning models to forget information present in their training data, provides us with a way to protect private data after training. Although several methods exist for such unlearning, it is unclear to what extent they result in models equivalent to those where the data to be forgotten was never learned in the first place. To address this challenge, we present TOFU, a Task of Fictitious Unlearning, as a benchmark aimed at helping deepen our understanding of unlearning. We offer a dataset of 200 diverse synthetic author profiles, each consisting of 20 question-answer pairs, and a subset of these profiles called the forget set that serves as the target for unlearning. We compile a suite of metrics that work together to provide a holistic picture of unlearning efficacy. Finally, we provide a set of baseline results from existing unlearning algorithms. Importantly, none of the baselines we consider show effective unlearning motivating continued efforts to develop approaches for unlearning that effectively tune models so that they truly behave as if they were never trained on the forget data at all.

Motivation & Objective

• To address the lack of standardized evaluation for unlearning in LLMs, especially for generative models.
• To create a well-defined, reproducible benchmark that isolates the impact of forgetting specific data.
• To develop a comprehensive evaluation framework combining forget quality and model utility metrics.
• To assess the effectiveness of existing unlearning algorithms under realistic computational constraints.
• To motivate future research by demonstrating that current methods fail to achieve meaningful unlearning.

Proposed method

• Creating a dataset of 200 fictitious author profiles, each with 20 question-answer pairs, to serve as training data for fine-tuning LLMs.
• Defining a 'forget set' of 2, 10, or 20 profiles to test unlearning at varying severity levels.
• Designing a forget quality metric based on the ratio of probabilities for correct vs. incorrect answers on the forget set.
• Applying a Kolmogorov-Smirnov (KS) test to compare the distribution of answer probabilities between unlearned and gold-standard retain models.
• Aggregating model utility across multiple evaluation datasets with varying relevance gradients to measure performance retention.
• Constraining unlearning algorithms to O(number of forget samples) compute to reflect real-world feasibility.

Experimental results

Research questions

• RQ1 Can existing unlearning methods effectively erase information about specific entities from fine-tuned LLMs?
• RQ2 To what extent do unlearning methods preserve model utility on non-forget data?
• RQ3 How well do current unlearning metrics correlate with approximate unlearning guarantees?
• RQ4 Can a unified benchmark like TOFU reveal systemic flaws in current unlearning approaches?
• RQ5 Is it possible to achieve effective unlearning without degrading overall model performance?

Key findings

• None of the four baseline unlearning methods tested achieved effective unlearning, as measured by both forget quality and model utility metrics.
• Many models exhibited high forget quality (low KS-test p-values) but poor model utility, indicating they produced gibberish or random outputs.
• The results show that current unlearning methods fail to achieve (ε,δ)-unlearning, even when they appear to forget on paper.
• The benchmark reveals that unlearning is fundamentally hard due to entanglement between learning and forgetting processes.
• Existing alignment techniques such as RLHF or DPO may not fully remove unwanted behaviors or memorized facts, suggesting a deeper issue in model control.
• The study demonstrates that current unlearning methods are weak attempts at privacy preservation, leaving significant room for improvement.