[Paper Review] Benchmarking Differential Privacy and Federated Learning for BERT Models

Priyam Basu, Tiasa Singha Roy+4 more|arXiv (Cornell University)|Jun 26, 2021

Privacy-Preserving Technologies in Data37 references6 citations

TL;DR

This paper benchmarks differential privacy (DP) and federated learning (FL) for BERT-based models in mental health NLP applications, evaluating privacy-utility trade-offs across BERT, RoBERTa, ALBERT, and DistilBERT on depression and sexual harassment Twitter datasets. Results show smaller models like ALBERT degrade more gracefully under DP, while FL with non-IID data incurs higher utility loss, and DP-FL combines privacy benefits with measurable performance trade-offs depending on model size and data distribution.

ABSTRACT

Natural Language Processing (NLP) techniques can be applied to help with the diagnosis of medical conditions such as depression, using a collection of a person's utterances. Depression is a serious medical illness that can have adverse effects on how one feels, thinks, and acts, which can lead to emotional and physical problems. Due to the sensitive nature of such data, privacy measures need to be taken for handling and training models with such data. In this work, we study the effects that the application of Differential Privacy (DP) has, in both a centralized and a Federated Learning (FL) setup, on training contextualized language models (BERT, ALBERT, RoBERTa and DistilBERT). We offer insights on how to privately train NLP models and what architectures and setups provide more desirable privacy utility trade-offs. We envisage this work to be used in future healthcare and mental health studies to keep medical history private. Therefore, we provide an open-source implementation of this work.

Motivation & Objective

• To evaluate the impact of differential privacy (DP) and federated learning (FL) on the utility of BERT-based models in mental health applications.
• To compare privacy-utility trade-offs across different model architectures (BERT, RoBERTa, ALBERT, DistilBERT) under centralized and federated training with DP.
• To analyze the effect of data distribution (IID vs. non-IID) on model performance in FL setups with DP.
• To provide empirical insights for designing privacy-preserving NLP systems in healthcare, especially for sensitive mental health data.
• To release an open-source framework to support reproducible benchmarking of DP and FL in NLP.

Proposed method

• The study uses Opacus to implement differential privacy with noise injection during model training across four BERT-based architectures.
• Centralized training with DP is evaluated across varying privacy budgets (ε = 0.5, 5, 15, ∞) on two Twitter datasets: depression and sexual harassment.
• Federated learning is implemented using FedAvg with 10 clients, each holding a shard of data, and DP is applied locally at each client per training round.
• Two FL data distribution scenarios are evaluated: IID (uniformly distributed data) and non-IID (non-uniform, realistic distribution), with 240 samples per client.
• Model performance is measured via test accuracy on held-out test sets, with results averaged over three random seeds and reported with standard deviation.
• An open-source framework is released to enable reproducibility and future benchmarking of DP and FL in NLP.

Experimental results

Research questions

• RQ1 How does differential privacy affect the utility of BERT-based models in centralized training for mental health text classification?
• RQ2 How do different model architectures (e.g., BERT vs. ALBERT) compare in terms of privacy-utility trade-offs under DP?
• RQ3 What is the impact of data distribution (IID vs. non-IID) on model utility in federated learning with and without DP?
• RQ4 How does combining DP with federated learning (DP-FL) affect model performance compared to DP or FL alone?
• RQ5 What are the optimal privacy budgets (ε) that balance utility and privacy for NLP models in healthcare applications?

Key findings

• Smaller models like ALBERT and DistilBERT degrade more gracefully under differential privacy compared to larger models like BERT and RoBERTa, showing better privacy-utility trade-offs.
• In non-IID FL settings, which reflect realistic medical data distributions, utility degradation is on average higher than in IID settings, indicating a need for tailored training algorithms.
• For the depression dataset, RoBERTa achieves the highest baseline accuracy (83.56 ± 0.00) without DP, while ALBERT shows the smallest performance drop when DP is applied.
• When combining DP with FL, the highest test accuracy (72.44 ± 1.74) is achieved with BERT in the non-IID FL setup, though this is still below the non-private FL baseline of 76.28 ± 0.11.
• As ε increases from 0.5 to 15, the standard deviation of test accuracy decreases, indicating more stable performance closer to the non-private baseline.
• With limited training data, the negative impact of DP on model utility is more pronounced, especially in low-data regimes, confirming prior findings on data sensitivity to DP noise.