[Paper Review] Generalization Techniques Empirically Outperform Differential Privacy against Membership Inference

Jiaxiang Liu, Simon Oya+1 more|arXiv (Cornell University)|Oct 11, 2021

Privacy-Preserving Technologies in Data20 references4 citations

TL;DR

This paper challenges the assumption that differential privacy (DP) is the most effective defense against membership inference attacks (MIAs), demonstrating that generalization techniques like Sharpness-Aware Minimization (SAM) and early stopping outperform DP-SGD in both privacy and utility. Surprisingly, even zero-noise DP-SGD performs worse than non-private generalization methods, which achieve higher accuracy and better protection against MIAs by reducing overfitting.

ABSTRACT

Differentially private training algorithms provide protection against one of the most popular attacks in machine learning: the membership inference attack. However, these privacy algorithms incur a loss of the model's classification accuracy, therefore creating a privacy-utility trade-off. The amount of noise that differential privacy requires to provide strong theoretical protection guarantees in deep learning typically renders the models unusable, but authors have observed that even lower noise levels provide acceptable empirical protection against existing membership inference attacks. In this work, we look for alternatives to differential privacy towards empirically protecting against membership inference attacks. We study the protection that simply following good machine learning practices (not designed with privacy in mind) offers against membership inference. We evaluate the performance of state-of-the-art techniques, such as pre-training and sharpness-aware minimization, alone and with differentially private training algorithms, and find that, when using early stopping, the algorithms without differential privacy can provide both higher utility and higher privacy than their differentially private counterparts. These findings challenge the belief that differential privacy is a good defense to protect against existing membership inference attacks

Motivation & Objective

• To evaluate whether generalization techniques—designed for model accuracy, not privacy—can empirically protect against membership inference attacks (MIAs).
• To compare the privacy-utility trade-off of DP-SGD against non-private generalization methods such as SAM, pre-training, and early stopping.
• To investigate whether combining DP noise with advanced optimization techniques like SAM improves privacy without sacrificing utility.
• To challenge the prevailing belief that DP is the optimal defense for MIAs, especially given its high utility cost.

Proposed method

• The authors evaluate state-of-the-art generalization techniques—SAM, pre-training on ImageNet, and early stopping—on CIFAR-10 and CIFAR-100 datasets.
• They compare these methods against DP-SGD with varying noise levels (σ), clipping thresholds (C), and learning rates (λ), using the Opacus privacy engine for DP training.
• The models are trained with and without DP noise, and performance is measured using both classification accuracy and membership inference attack success rates.
• Two standard MIAs are used: the threshold attack (Yeom et al., 2018) and the neural network-based attack (Salem et al., 2018), with results reported at each training epoch.
• The study includes ablation experiments on DP-SAM, a hybrid of SAM and DP-SGD, to assess whether combining both methods improves the privacy-utility trade-off.
• All experiments are conducted with early stopping, and the privacy-utility performance is evaluated across multiple training epochs to assess dynamic behavior.

Experimental results

Research questions

• RQ1 Does training with generalization techniques like SAM and early stopping provide better empirical protection against membership inference attacks than DP-SGD?
• RQ2 What is the privacy-utility trade-off of DP-SGD when noise levels are reduced to zero, and how does it compare to non-private generalization methods?
• RQ3 Can combining SAM with DP-SGD (DP-SAM) improve privacy without degrading utility compared to standard DP-SGD?
• RQ4 How do pre-training on ImageNet and other standard ML practices affect the model’s resistance to membership inference attacks?
• RQ5 Are there specific training phases (e.g., early stopping) where non-private models outperform DP-regularized models in both accuracy and privacy?

Key findings

• Zero-noise DP-SGD consistently underperforms non-private generalization techniques in both privacy and utility, indicating that DP noise provides no benefit and may even harm performance.
• SAM outperforms DP-SGD in both privacy and utility, achieving higher test accuracy and lower membership inference attack success rates across both CIFAR-10 and CIFAR-100.
• Early stopping without any noise provides better privacy-utility trade-offs than DP-SGD with any noise level, suggesting that overfitting reduction is more effective than noise injection.
• The combination of SAM and DP-SGD (DP-SAM) does not improve privacy over non-DP SAM, and the non-private version remains superior in both accuracy and attack resistance.
• Out of all evaluated methods, non-private SAM with early stopping achieves the best balance of high accuracy and low membership inference success rate, especially for outlier samples.
• The results show that generalization techniques that reduce overfitting inherently improve privacy, challenging the necessity of DP for empirical MIA defense.