[Paper Review] Towards Measuring Membership Privacy

Yunhui Long, Vincent Bindschaedler+1 more|arXiv (Cornell University)|Dec 25, 2017

Privacy-Preserving Technologies in Data30 references63 citations

TL;DR

The paper introduces Differential Training Privacy (DTP) as an empirical metric to quantify membership inference risk for classifiers when differential privacy cannot be applied, and proposes PDTP as an efficiently computable lower bound. It shows DTP/PDTP predict membership attack success and advocates using DTP-1 as a publishing guideline.

ABSTRACT

Machine learning models are increasingly made available to the masses through public query interfaces. Recent academic work has demonstrated that malicious users who can query such models are able to infer sensitive information about records within the training data. Differential privacy can thwart such attacks, but not all models can be readily trained to achieve this guarantee or to achieve it with acceptable utility loss. As a result, if a model is trained without differential privacy guarantee, little is known or can be said about the privacy risk of releasing it. In this work, we investigate and analyze membership attacks to understand why and how they succeed. Based on this understanding, we propose Differential Training Privacy (DTP), an empirical metric to estimate the privacy risk of publishing a classier when methods such as differential privacy cannot be applied. DTP is a measure of a classier with respect to its training dataset, and we show that calculating DTP is efficient in many practical cases. We empirically validate DTP using state-of-the-art machine learning models such as neural networks trained on real-world datasets. Our results show that DTP is highly predictive of the success of membership attacks and therefore reducing DTP also reduces the privacy risk. We advocate for DTP to be used as part of the decision-making process when considering publishing a classifier. To this end, we also suggest adopting the DTP-1 hypothesis: if a classifier has a DTP value above 1, it should not be published.

Motivation & Objective

• Motivate and quantify privacy risks of publishing classifiers exposed to public queries without DP guarantees.
• Develop an empirical, classifier- and dataset-specific privacy metric (DTP) to measure membership leakage.
• Introduce PDTP as an efficiently computable lower bound to DTP and connect it to direct membership attacks.
• Validate DTP/PDTP on real datasets and common models to guide publishing decisions for MLaaS.
• Propose the DTP-1 hypothesis as a practical publishing threshold.

Proposed method

• Define and formalize Differential Training Privacy (DTP) as a bound on how predictions change when a training record is left out.
• Propose PDTP as a computationally efficient proxy for DTP using leave-one-out leave-one-record-out evaluation.
• Construct a general membership attack framework (untargeted, distance-based, frequency-based) and shadow-model based attacks to evaluate privacy.
• Evaluate on real datasets (UCI Adult and NN-Purchase) and multiple models (NN, NB, LR) with prediction binning to stabilize measurements.
• Analyze training stability and provide theoretical insights on when direct attacks dominate indirect attacks.

Experimental results

Research questions

• RQ1 Can we quantify membership privacy risk of a classifier using an empirical, non-DP metric like DTP?
• RQ2 Is PDTP a reliable, efficient lower bound for DTP that correlates with actual membership attack success?
• RQ3 Does the DTP-1 threshold (DTP > 1 implies not publishing) hold across datasets and models?
• RQ4 How does classifier overfitting or training stability affect susceptibility to membership inference attacks?
• RQ5 What is the relationship between attack type (untargeted, distance-based, frequency-based) and PDTP/DTP measures?

Key findings

• DTP values correlate strongly with membership attack success across experiments (e.g., NN-Purchase: r = 0.7653 for distance-based attack; overall strong attack correlations).
• For DTP-values below 0.5, attacks did not infer membership with accuracy above baseline; when DTP exceeds 4, attacks frequently exceed 90% accuracy.
• PDTP provides a lower bound to DTP and serves as an efficient indicator of membership privacy risk via leave-one-out evaluation.
• The three analyzed direct attacks show varying performance, with distance-based attacks achieving the highest accuracy (e.g., 0.8533) and strongest correlation with PDTP.
• The study supports the DTP-1 hypothesis as a practical guideline: classifiers with DTP > 1 should not be published.
• Training stability is identified as a key factor; Naive Bayes, random decision trees, and linear statistical queries satisfy training stability, while k-NN does not.