[論文レビュー] Deduplicating Training Data Mitigates Privacy Risks in Language Models

Nikhil Kandpal, Eric Wallace+1 more|arXiv (Cornell University)|Feb 14, 2022

Privacy-Preserving Technologies in Data被引用数 45

ひとこと要約

本論文は、ウェブスクレイピングで得られた訓練データの重複が記憶化とプライバシー攻撃の成功につながることを示している。データの重複を排除することは、言語モデルの性能を損なうことなく、リークを大幅に低減する。

ABSTRACT

Past work has shown that large language models are susceptible to privacy attacks, where adversaries generate sequences from a trained model and detect which sequences are memorized from the training set. In this work, we show that the success of these attacks is largely due to duplication in commonly used web-scraped training sets. We first show that the rate at which language models regenerate training sequences is superlinearly related to a sequence's count in the training set. For instance, a sequence that is present 10 times in the training data is on average generated ~1000 times more often than a sequence that is present only once. We next show that existing methods for detecting memorized sequences have near-chance accuracy on non-duplicated training sequences. Finally, we find that after applying methods to deduplicate training data, language models are considerably more secure against these types of privacy attacks. Taken together, our results motivate an increased focus on deduplication in privacy-sensitive applications and a reevaluation of the practicality of existing privacy attacks.

研究の動機と目的

• Identify how exact sequence duplication in training data affects generation of training samples by language models.
• Assess the effectiveness of membership inference attacks under varying levels of data duplication.
• Evaluate whether deduplicating training data reduces privacy risks without harming model performance.

提案手法

• Measure regeneration rates of training sequences as a function of their duplicate counts in the training data.
• Adapt suffix-array based ExactSubstr deduplication to detect exact duplicates.
• Evaluate Carlini et al. style model inversion attacks and membership inference scores under different duplication settings.
• Retrain models on deduplicated data and compare leakage and AUROC of membership inference methods.
• Provide a table of results showing leakage and AUROC for normal vs deduplicated models.

実験結果

リサーチクエスチョン

• RQ1 How does the number of duplicates of a training sequence affect its likelihood of being regenerated by an LM?
• RQ2 How effective are membership inference methods at detecting memorized sequences at different duplication levels?
• RQ3 Does deduplicating training data meaningfully reduce privacy risk without degrading language model performance?

主な発見

Metric	Normal Model	Deduped Model
Training Data Generated Count	1,427,212	68,090
Percent	0.14	0.007
Mem. Inference AUROC (zlib)	0.76	0.67
Mem. Inference AUROC (Ref Model)	0.88	0.87
Mem. Inference AUROC (Lowercase)	0.86	0.68

• Regeneration is superlinearly related to training sequence duplicates; sequences duplicated 10 times are regenerated ~1000x more often than those duplicated once.
• Memorization detection methods perform near chance on non-duplicated sequences, and their effectiveness scales with duplication level.
• Deduplicated training data leads to ~20x less training data emitted by models and reduces AUROC of membership inference methods for most duplications; perplexity remains unaffected.
• When deduplicated models do regenerate training data, membership inference can still detect some leakage, especially with the Reference Model score.
• Overall, deduplication is an effective defense against model inversion attacks with minimal impact on model performance.