[Paper Review] Understanding Membership Inferences on Well-Generalized Learning Models

Yunhui Long, Vincent Bindschaedler+6 more|arXiv (Cornell University)|Feb 13, 2018

Adversarial Robustness in Machine Learning35 references164 citations

TL;DR

The paper shows that membership inference attacks can succeed on well-generalized models using generalized MIA (GMIA); it identifies vulnerable records, uses reference models, and demonstrates both direct and indirect inferences with real-data evaluations.

ABSTRACT

Membership Inference Attack (MIA) determines the presence of a record in a machine learning model's training data by querying the model. Prior work has shown that the attack is feasible when the model is overfitted to its training data or when the adversary controls the training algorithm. However, when the model is not overfitted and the adversary does not control the training algorithm, the threat is not well understood. In this paper, we report a study that discovers overfitting to be a sufficient but not a necessary condition for an MIA to succeed. More specifically, we demonstrate that even a well-generalized model contains vulnerable instances subject to a new generalized MIA (GMIA). In GMIA, we use novel techniques for selecting vulnerable instances and detecting their subtle influences ignored by overfitting metrics. Specifically, we successfully identify individual records with high precision in real-world datasets by querying black-box machine learning models. Further we show that a vulnerable record can even be indirectly attacked by querying other related records and existing generalization techniques are found to be less effective in protecting the vulnerable instances. Our findings sharpen the understanding of the fundamental cause of the problem: the unique influences the training instance may have on the model.

Motivation & Objective

• Reassess whether overfitting is the sole driver of membership disclosure in ML models.
• Develop a generalized membership inference attack (GMIA) capable of attacking well-generalized models.
• Identify vulnerable training records and small influences that reveal membership via black-box queries.
• Evaluate attack effectiveness on real-world datasets and contrast with regularization defenses.

Proposed method

• Introduce generalized membership inference attack (GMIA) that does not rely on shadow models for well-generalized targets.
• Build reference models via bootstrap sampling to mimic target model behavior on accessible data.
• Select vulnerable target records by measuring their unique influence using high-level features from reference models.
• Perform direct inference by comparing target model outputs to reference model distributions using a log-loss based hypothesis test.
• Enable indirect inference by identifying enhancing records and conducting queries on non-target records; combine results with Kost’s method.

Experimental results

Research questions

• RQ1 Is overfitting a necessary condition for membership disclosure in ML models?
• RQ2 Can membership inference succeed on well-generalized models without access to the training algorithm?
• RQ3 What causes membership leaks beyond overfitting, and can we detect vulnerable training records?
• RQ4 How effective is generalized MIA (GMIA) in both direct and indirect inference scenarios?
• RQ5 How do regularization techniques impact the success of GMIA on real datasets?

Key findings

• MIAs can succeed on well-generalized models, with high-precision attacks on selected vulnerable records.
• On MNIST, 73.88% of models yielded an attack precision of 93.36% for vulnerable objects.
• On Cancer, 3.2% of models yielded an attack precision of 88.89% for target patients.
• On Adult, 5.23% of models yielded an attack precision of 73.91% for target individuals.
• Regularization (e.g., L2) does not reliably eliminate the threat; e.g., MNIST inference remains possible in 34% of models with 100% precision under certain settings.
• Indirect inferences can outperform direct inferences in some cases, including 100% precision in 16% of models for Adult dataset.