[Paper Review] Learning Differentially Private Recurrent Language Models

H. Brendan McMahan, Daniel Ramage+2 more|arXiv (Cornell University)|Oct 18, 2017

Privacy-Preserving Technologies in Data670 citations

TL;DR

The authors train large LSTM language models with user-level differential privacy using a noised Federated Averaging approach and moments accountant, achieving similar accuracy to non-private models at a substantial computational cost, with privacy improving on larger datasets.

ABSTRACT

We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees with only a negligible cost in predictive accuracy. Our work builds on recent advances in the training of deep networks on user-partitioned data and privacy accounting for stochastic gradient descent. In particular, we add user-level privacy protection to the federated averaging algorithm, which makes "large step" updates from user-level data. Our work demonstrates that given a dataset with a sufficiently large number of users (a requirement easily met by even small internet-scale datasets), achieving differential privacy comes at the cost of increased computation, rather than in decreased utility as in most prior work. We find that our private LSTM language models are quantitatively and qualitatively similar to un-noised models when trained on a large dataset.

Motivation & Objective

• Provide strong user-level differential privacy guarantees for recurrent language models.
• Extend Federated Averaging with noise and clipping to achieve DP at user level.
• Demonstrate that private LSTMs can match non-private performance on large datasets.
• Offer practical guidance on parameter tuning for private training of complex models.

Proposed method

• Apply user-adjacent differential privacy to model training.
• Introduce a noised version of Federated Averaging (DP-FedAvg) with random user sampling, per-user update clipping, and Gaussian noise.
• Use two bounded-sensitivity estimators for weighted averages during aggregation (tilde{f}_f and tilde{f}_c).
• Clip per-user updates to bound L2 norm before aggregation.
• Add Gaussian noise scaled to the estimated sensitivity and use the Moments Accountant to bound privacy loss.
• Leverage amplification via sampling to obtain tighter DP guarantees on large datasets.

Experimental results

Research questions

• RQ1 Can user-level differential privacy be practically applied to training large recurrent language models without prohibitive utility loss?
• RQ2 How does DP-FedAvg perform in terms of accuracy versus privacy trade-offs across different clipping and noise settings?
• RQ3 What is the impact of dataset size on achievable privacy guarantees and model utility?
• RQ4 What practical guidelines emerge for tuning clipping bounds and noise in private training of LSTMs?

Key findings

• Private LSTM language models can achieve near-non-private accuracy on large datasets with strong user-level DP.
• On a dataset of 763,430 users, non-private training achieves 17.5% accuracy in 4120 rounds, while private training with (4.6, 1e-9) DP reaches similar accuracy in 4980 rounds (processing ~5000 users per round) at ~60x computational cost.
• Increasing dataset size to around 1e8 users can improve privacy to (1.2, 1e-9) with similar utility under the same framework.
• DP-FedAvg enables training of complex models (embedding + dense state transitions) under DP without large utility loss, though at the cost of computation.
• Empirical results provide guidelines for parameter tuning, showing that privacy costs can be driven more by computation than by utility loss once the dataset is large enough.