[Paper Review] ML Privacy Meter: Aiding Regulatory Compliance by Quantifying the Privacy Risks of Machine Learning

Sasi Kumar Murakonda, Reza Shokri|arXiv (Cornell University)|Jul 18, 2020

Privacy-Preserving Technologies in Data7 references38 citations

TL;DR

ML Privacy Meter quantifies the privacy risk to training data from ML models using membership inference attacks, aiding DPIA and regulatory compliance.

ABSTRACT

When building machine learning models using sensitive data, organizations should ensure that the data processed in such systems is adequately protected. For projects involving machine learning on personal data, Article 35 of the GDPR mandates it to perform a Data Protection Impact Assessment (DPIA). In addition to the threats of illegitimate access to data through security breaches, machine learning models pose an additional privacy risk to the data by indirectly revealing about it through the model predictions and parameters. Guidances released by the Information Commissioner's Office (UK) and the National Institute of Standards and Technology (US) emphasize on the threat to data from models and recommend organizations to account for and estimate these risks to comply with data protection regulations. Hence, there is an immediate need for a tool that can quantify the privacy risk to data from models. In this paper, we focus on this indirect leakage about training data from machine learning models. We present ML Privacy Meter, a tool that can quantify the privacy risk to data from models through state of the art membership inference attack techniques. We discuss how this tool can help practitioners in compliance with data protection regulations, when deploying machine learning models.

Motivation & Objective

• Motivate the need to assess privacy risks from ML models beyond traditional security threats.
• Propose a quantitative tool to measure indirect data leakage through model predictions and parameters.
• Show how the tool supports regulatory compliance by informing risk assessment and mitigation decisions.

Proposed method

• Implement membership inference attack techniques to assess privacy risk under different access assumptions (black-box, white-box).
• Compute risk scores for training records by comparing attack output distributions for members vs non-members.
• Use ROC curves to quantify attacker success and derive an aggregate privacy risk measure (area under the curve).
• Generate detailed privacy reports that compare risk across records and access levels (predictions only vs predictions plus parameters).
• Provide guidance on mitigation strategies and enable evaluation of differential privacy parameters (epsilon) in practice.

Experimental results

Research questions

• RQ1 How can we quantify the privacy risk to training data from ML models using membership inference attacks?
• RQ2 How does access level (black-box vs white-box) affect the inferred privacy risk?
• RQ3 Can the tool help select mitigation strategies and privacy parameters to balance utility and privacy?

Key findings

• The tool quantifies privacy risk via membership inference attack success, using ROC curves to represent trade-offs between true and false positives.
• It provides privacy risk scores for all training records and allows comparison across classes.
• It enables evaluation of privacy risk with different access modes (black-box vs white-box) and supports reporting at aggregate and per-record levels.
• ML Privacy Meter can guide practical mitigation actions and inform DPIA by identifying high-risk data records.
• The framework supports exploring differential privacy guarantees and selecting epsilon values by measuring risk at each epsilon.