[Paper Review] Large Language Models Can Be Strong Differentially Private Learners

Xuechen Li, Florian Tramèr+2 more|arXiv (Cornell University)|Oct 12, 2021

Privacy-Preserving Technologies in DataComputer Science79 references73 citations

TL;DR

The paper shows that fine-tuning large pretrained language models with DP optimization (DP-SGD/DP-Adam) can achieve strong performance on NLP tasks under modest privacy budgets, aided by a memory-efficient ghost clipping technique and task-aligned objectives.

ABSTRACT

Differentially Private (DP) learning has seen limited success for building large deep learning models of text, and straightforward attempts at applying Differentially Private Stochastic Gradient Descent (DP-SGD) to NLP tasks have resulted in large performance drops and high computational overhead. We show that this performance drop can be mitigated with (1) the use of large pretrained language models; (2) non-standard hyperparameters that suit DP optimization; and (3) fine-tuning objectives which are aligned with the pretraining procedure. With the above, we obtain NLP models that outperform state-of-the-art DP-trained models under the same privacy budget and strong non-private baselines -- by directly fine-tuning pretrained models with DP optimization on moderately-sized corpora. To address the computational challenge of running DP-SGD with large Transformers, we propose a memory saving technique that allows clipping in DP-SGD to run without instantiating per-example gradients for any linear layer in the model. The technique enables privately training Transformers with almost the same memory cost as non-private training at a modest run-time overhead. Contrary to conventional wisdom that DP optimization fails at learning high-dimensional models (due to noise that scales with dimension) empirical results reveal that private learning with pretrained language models doesn't tend to suffer from dimension-dependent performance degradation. Code to reproduce results can be found at https://github.com/lxuechen/private-transformers.

Motivation & Objective

• Motivate differential privacy in NLP and address performance/compute bottlenecks for large models under DP.
• Demonstrate that large pretrained language models can be effectively fine-tuned with DP optimization within reasonable privacy budgets (epsilon in {3,8}).
• Identify hyperparameter choices and task objectives that align with DP optimization to maximize performance.
• Develop memory-efficient techniques to enable DP training of Transformers without per-example gradients (ghost clipping).

Proposed method

• Fine-tune pretrained models (BERT/RoBERTa family for classification; GPT-2 variants for generation) with DP-Adam under (epsilon, delta) privacy accounting.
• Systematically study hyperparameters (batch size, learning rate, training epochs) and their impact on DP performance across tasks.
• Use text-infilling style fine-tuning for classification to align with pretraining objectives.
• Develop ghost clipping to avoid instantiating per-example gradients in DP-SGD for Transformers and sequential data.
• Apply layer-by-layer clipping and sequential-friendly gradient-norm computations to reduce memory usage.
• Compare full fine-tuning with low-dimensional update methods and report privacy-bounded performance across tasks.

Experimental results

Research questions

• RQ1 Can large pretrained language models be effectively fine-tuned under differential privacy to achieve competitive NLP performance?
• RQ2 How do hyperparameters and task-aligned objectives influence DP optimization performance on classification and generation tasks?
• RQ3 Does the dimensionality of parameter updates limit DP performance, or can full fine-tuning outperform low-dimensional approaches under DP?
• RQ4 Can memory-efficient techniques like ghost clipping enable private training of large Transformer models at practical memory and compute costs?
• RQ5 What is the impact of model size on private fine-tuning performance under fixed privacy budgets?

Key findings

• Pretrained models fine-tuned with DP-Adam achieve strong performance under privacy budgets ε in {3,8} across tasks.
• Larger pretrained models yield better private fine-tuning results.
• Full fine-tuning with DP-Adam matches or exceeds specialized low-dimensional DP methods on classification and generation.
• Ghost clipping reduces memory usage dramatically and enables privately training Transformers with near non-private memory costs, with modest runtime overhead.
• Task alignment, such as text-infilling for classification, improves private learning performance.
• On generation tasks, private models can surpass strong non-private baselines under DP.