[Paper Review] Adaptive Machine Unlearning

Varun Gupta, Christopher Jung+4 more|arXiv (Cornell University)|Jun 8, 2021

Privacy-Preserving Technologies in DataComputer Science25 references45 citations

TL;DR

The paper introduces a generic reduction to enable adaptive deletion guarantees for unlearning in machine learning, combining differential privacy with max-information to handle adaptive deletion sequences across arbitrary model classes and training methods, and demonstrates limitations of prior non-adaptive methods via theory and experiments.

ABSTRACT

Data deletion algorithms aim to remove the influence of deleted data points from trained models at a cheaper computational cost than fully retraining those models. However, for sequences of deletions, most prior work in the non-convex setting gives valid guarantees only for sequences that are chosen independently of the models that are published. If people choose to delete their data as a function of the published models (because they don't like what the models reveal about them, for example), then the update sequence is adaptive. In this paper, we give a general reduction from deletion guarantees against adaptive sequences to deletion guarantees against non-adaptive sequences, using differential privacy and its connection to max information. Combined with ideas from prior work which give guarantees for non-adaptive deletion sequences, this leads to extremely flexible algorithms able to handle arbitrary model classes and training methodologies, giving strong provable deletion guarantees for adaptive deletion sequences. We show in theory how prior work for non-convex models fails against adaptive deletion sequences, and use this intuition to design a practical attack against the SISA algorithm of Bourtoule et al. [2021] on CIFAR-10, MNIST, Fashion-MNIST.

Motivation & Objective

• Motivate data deletion (unlearning) under regulatory and security pressures and reduce retraining cost.
• Provide guarantees against adaptive sequences of deletion requests that depend on published models.
• Develop a modular framework that leverages DP and max-information to extend non-adaptive guarantees to adaptive settings.
• Show limitations of existing non-adaptive methods and propose a practical, DP-driven distributed unlearning approach.
• Demonstrate theoretical and empirical results validating adaptive deletion guarantees across datasets and model families.

Proposed method

• Define a formal unlearning framework with alpha, beta, gamma guarantees for adaptive deletion sequences.
• Propose a generic reduction: if a learning/unlearning pair has non-adaptive deletion guarantees and publishing is differentially private in internal randomness, then adaptive guarantees hold.
• Adopt a distributed “SISA-like” architecture with independently trained shards and DP-based aggregation to enable adaptive guarantees.
• Use differential privacy to bound max-information between internal randomness and updates, enabling control over adaptive adversaries.
• Show how private aggregation and DP budget management yield adaptive guarantees and practical retraining triggers.
• Provide theoretical analysis and experimental validation on CIFAR-10, MNIST, and Fashion-MNIST.

Experimental results

Research questions

• RQ1 How can deletion guarantees be extended from non-adaptive to adaptive sequences of deletion requests?
• RQ2 Can differential privacy and max-information be leveraged to provide robust adaptive unlearning guarantees across arbitrary model classes?
• RQ3 What are the limitations of existing non-adaptive deletion methods (e.g., SISA) when faced with adaptive deletion sequences?
• RQ4 How can a distributed/unlearning framework be designed to maintain performance while offering provable adaptive deletion guarantees?

Key findings

• A generic reduction shows adaptive deletion guarantees follow from non-adaptive guarantees plus publishing DP in internal randomness.
• SISA-like distributed architectures can achieve adaptive guarantees when shard data are independently selected and randomness is DP-protected.
• Prior non-convex deletion methods may fail under adaptive deletion sequences, illustrating the need for DP-based reductions.
• Experiments reveal adaptive deletion weaknesses in SISA under label-only and white-box settings, with DP providing mitigation beyond theoretical guarantees.
• Private aggregation and DP budget management enable practical adaptive unlearning with acceptable accuracy trade-offs compared to full retraining.