[Paper Review] ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models

Yugeng Liu, Rui Wen+7 more|arXiv (Cornell University)|Feb 4, 2021

Adversarial Robustness in Machine LearningComputer Science67 references46 citations

TL;DR

ML-Doctor offers a holistic, modular framework to assess privacy risks from membership inference, model inversion, attribute inference, and model stealing across multiple architectures and datasets, plus defenses like DP-SGD and knowledge distillation.

ABSTRACT

Inference attacks against Machine Learning (ML) models allow adversaries to learn sensitive information about training data, model parameters, etc. While researchers have studied, in depth, several kinds of attacks, they have done so in isolation. As a result, we lack a comprehensive picture of the risks caused by the attacks, e.g., the different scenarios they can be applied to, the common factors that influence their performance, the relationship among them, or the effectiveness of possible defenses. In this paper, we fill this gap by presenting a first-of-its-kind holistic risk assessment of different inference attacks against machine learning models. We concentrate on four attacks -- namely, membership inference, model inversion, attribute inference, and model stealing -- and establish a threat model taxonomy. Our extensive experimental evaluation, run on five model architectures and four image datasets, shows that the complexity of the training dataset plays an important role with respect to the attack's performance, while the effectiveness of model stealing and membership inference attacks are negatively correlated. We also show that defenses like DP-SGD and Knowledge Distillation can only mitigate some of the inference attacks. Our analysis relies on a modular re-usable software, ML-Doctor, which enables ML model owners to assess the risks of deploying their models, and equally serves as a benchmark tool for researchers and practitioners.

Motivation & Objective

• Provide a comprehensive taxonomy of threat models for inference attacks on ML models.
• Quantify how dataset complexity and model overfitting influence attack performance.
• Explore relationships among different inference attacks and defenses across architectures and datasets.
• Deliver a modular, reusable framework (ML-Doctor) to benchmark attacks and defenses for researchers and model owners.

Proposed method

• Define a two-dimensional threat model taxonomy (model access: white-box/black-box; auxiliary data: partial/shadow/none).
• Formalize four inference attacks (membership inference, model inversion, attribute inference, model stealing) under various threat models.
• Conduct extensive empirical evaluation on five model architectures and four image datasets, analyzing attack performance and defense effectiveness.
• Implement ML-Doctor as a modular framework with data processing, attack, defense, and evaluation modules.
• Use shadow models and auxiliary data to train attack models for membership inference and related attacks.
• Assess defenses such as DP-SGD and Knowledge Distillation across attacks to determine defense coverage and limitations.

Experimental results

Research questions

• RQ1 RQ1: What is the impact of dataset complexity on different attacks?
• RQ2 RQ2: What is the impact of overfitting on different attacks?
• RQ3 RQ3: What is the relationship among different attacks?

Key findings

• Dataset complexity strongly affects membership inference, model inversion, and model stealing; membership inference benefits from more complex datasets, while the opposite is often true for model stealing.
• There is a negative correlation between membership inference success and model stealing success (r = -0.821) driven by overfitting effects.
• White-box access generally yields stronger attack performance than black-box access across attacks.
• DP-SGD can mitigate membership inference with limited impact on model utility; Knowledge Distillation helps but is less effective for some attacks.
• Partial auxiliary data does not significantly improve attack performance for membership inference, attribute inference, or model stealing across evaluated settings.
• Model stealing achieves higher agreement on simpler datasets (e.g., FMNIST) than on complex ones (e.g., STL10) due to overfitting dynamics.