[Paper Review] Effects of Differential Privacy and Data Skewness on Membership Inference Vulnerability

Stacey Truex, Ling Liu+3 more|arXiv (Cornell University)|Nov 21, 2019

Privacy-Preserving Technologies in Data41 references4 citations

TL;DR

This paper introduces MPLens, a system for analyzing membership inference vulnerability in machine learning models, demonstrating that data skewness increases risk for minority classes and that differential privacy introduces significant trade-offs in accuracy and privacy protection. The study reveals that skewed datasets amplify membership inference threats, and while differential privacy reduces vulnerability, it often degrades model performance.

ABSTRACT

Membership inference attacks seek to infer the membership of individual training instances of a privately trained model. This paper presents a membership privacy analysis and evaluation system, called MPLens, with three unique contributions. First, through MPLens, we demonstrate how membership inference attack methods can be leveraged in adversarial machine learning. Second, through MPLens, we highlight how the vulnerability of pre-trained models under membership inference attack is not uniform across all classes, particularly when the training data itself is skewed. We show that risk from membership inference attacks is routinely increased when models use skewed training data. Finally, we investigate the effectiveness of differential privacy as a mitigation technique against membership inference attacks. We discuss the trade-offs of implementing such a mitigation strategy with respect to the model complexity, the learning task complexity, the dataset complexity and the privacy parameter settings. Our empirical results reveal that (1) minority groups within skewed datasets display increased risk for membership inference and (2) differential privacy presents many challenging trade-offs as a mitigation technique to membership inference risk.

Motivation & Objective

• To investigate how data skewness affects membership inference vulnerability in machine learning models.
• To evaluate the effectiveness of differential privacy as a mitigation technique against membership inference attacks.
• To develop a systematic framework, MPLens, for assessing membership privacy risks in MLaaS platforms.
• To explore the interplay between model complexity, learning task complexity, and privacy parameter settings in membership inference attacks.
• To provide insights for privacy-preserving federated learning by analyzing differential privacy's impact on membership leakage.

Proposed method

• The authors design MPLens, a membership privacy analysis and evaluation system for black-box membership inference attacks.
• They use shadow model attacks to generate substitute models that mimic the behavior of target models, enabling membership inference on the target.
• The system evaluates models trained on skewed datasets to measure class-specific membership inference vulnerability.
• Differential privacy is applied by adding calibrated noise to model updates during training to protect membership privacy.
• The method evaluates trade-offs across model complexity, task complexity, dataset complexity, and privacy budget (ε) settings.
• Empirical evaluation is conducted across multiple datasets and deep learning architectures to assess attack success rates and model utility.

Experimental results

Research questions

• RQ1 How does data skewness in training sets affect the vulnerability of individual classes to membership inference attacks?
• RQ2 To what extent does differential privacy reduce membership inference vulnerability in deep learning models?
• RQ3 What are the trade-offs between model accuracy and privacy protection when applying differential privacy?
• RQ4 How do model complexity and learning task complexity influence membership inference risk and mitigation effectiveness?
• RQ5 Can membership inference attacks be leveraged to generate substitute models that facilitate adversarial attacks?

Key findings

• Minority classes in skewed datasets exhibit significantly higher membership inference vulnerability compared to majority classes.
• The use of differential privacy reduces membership inference success rates but often results in substantial drops in model accuracy.
• The effectiveness of differential privacy as a mitigation technique is highly sensitive to the choice of privacy budget (ε), with lower ε values increasing privacy but decreasing model utility.
• Model complexity and learning task complexity amplify the trade-offs between privacy and performance when applying differential privacy.
• Data skewness leads to non-uniform privacy risks, where underrepresented classes are disproportionately exposed to membership inference threats.
• The study demonstrates that membership inference can be used to generate high-fidelity substitute models, increasing the risk of adversarial attacks.