[Paper Review] On Inferring Training Data Attributes in Machine Learning Models

Benjamin Zi Hao Zhao, Hassan Jameel Asghar+2 more|arXiv (Cornell University)|Aug 28, 2019

Adversarial Robustness in Machine Learning10 references6 citations

TL;DR

This paper challenges the assumption that membership inference attacks (MIAs) can reliably enable attribute inference attacks (AIAs) by demonstrating that MIAs fail to distinguish between training data members and similar non-members. The authors propose a relaxed 'approximate attribute inference' model, showing that while MIAs perform poorly on close neighbors, they still enable meaningful attribute recovery when the goal is to find nearby, plausible attribute values rather than exact matches.

ABSTRACT

A number of recent works have demonstrated that API access to machine learning models leaks information about the dataset records used to train the models. Further, the work of \cite{somesh-overfit} shows that such membership inference attacks (MIAs) may be sufficient to construct a stronger breed of attribute inference attacks (AIAs), which given a partial view of a record can guess the missing attributes. In this work, we show (to the contrary) that MIA may not be sufficient to build a successful AIA. This is because the latter requires the ability to distinguish between similar records (differing only in a few attributes), and, as we demonstrate, the current breed of MIA are unsuccessful in distinguishing member records from similar non-member records. We thus propose a relaxed notion of AIA, whose goal is to only approximately guess the missing attributes and argue that such an attack is more likely to be successful, if MIA is to be used as a subroutine for inferring training record attributes.

Motivation & Objective

• To investigate whether membership inference attacks (MIAs) can effectively support attribute inference attacks (AIAs) in machine learning models.
• To challenge the assumption that MIAs can reliably distinguish between training data members and similar non-members.
• To propose a relaxed form of attribute inference—'approximate attribute inference'—that focuses on identifying nearby, plausible attribute values rather than exact matches.
• To empirically evaluate the performance of MIAs in distinguishing members from non-members at varying Hamming distances.
• To assess the limitations of current MIA-based AIA approaches and explore alternative attack models.

Proposed method

• The authors define 'strong membership inference' as the task of distinguishing a member from a non-member that is close in feature space, using a given distance metric.
• They introduce 'approximate attribute inference' as a relaxed AIA variant, where the goal is to output a vector close to the true missing attributes, not necessarily the exact one.
• Experiments are conducted on neural network models trained on Location-30 and Purchases-2,10,20,50,100 datasets, using confidence scores from model outputs as the basis for inference.
• The authors generate synthetic non-member vectors at controlled Hamming distances from training members to test MIA performance across varying distances.
• They evaluate MIA success by measuring the distinguisher's accuracy in identifying members versus non-members at different distances, using the highest confidence score as the decision signal.
• The attack is evaluated by measuring the average Hamming distance between inferred vectors and true member vectors, comparing to random guessing.

Experimental results

Research questions

• RQ1 Can membership inference attacks reliably distinguish between a training data member and a non-member that is similar in feature space?
• RQ2 Does the performance of membership inference degrade significantly when the non-member is close to a training member?
• RQ3 Can attribute inference attacks be successfully mounted using membership inference as a subroutine, especially when the target record is similar to non-members?
• RQ4 Is there a viable alternative to exact attribute inference that remains effective when MIAs fail on close neighbors?
• RQ5 How does the Hamming distance between a member and a non-member affect the distinguishability of the two via confidence scores?

Key findings

• Membership inference attacks fail to distinguish between training data members and non-members that are close in feature space, with performance no better than random guessing when Hamming distance is small.
• The success of membership inference increases significantly as the Hamming distance between a non-member and the nearest training member increases.
• On the Location-30 dataset, the proposed approximate attribute inference attack achieved a mean Hamming distance of approximately 6.5 from the true member vector, significantly better than the 7.5 expected from random guessing.
• The distribution of inferred vectors in the attack shows a clear concentration near the true member vectors, indicating that the method effectively identifies plausible attribute completions.
• The results demonstrate that while MIA is ineffective for exact attribute inference due to confusion with similar non-members, it remains useful for approximate inference.
• The study shows that the standard assumption that MIA can be used as a subroutine for AIA is flawed when the number of missing attributes is small and the non-members are close in feature space.