[Paper Review] Inference Attacks Against Collaborative Learning.

Luca Melis, Congzheng Song+2 more|arXiv (Cornell University)|May 10, 2018

Adversarial Robustness in Machine Learning66 references93 citations

TL;DR

This paper demonstrates that collaborative learning systems are vulnerable to inference attacks, where adversarial participants can infer exact training data points (membership inference) and hidden data properties not captured by the joint model. The attacks exploit shared model parameters or gradients, showing high success rates across diverse tasks and datasets, highlighting critical privacy risks in federated and distributed learning.

ABSTRACT

Collaborative machine learning and related techniques such as distributed and federated learning allow multiple participants, each with his own training dataset, to build a joint model. Participants train local models and periodically exchange model parameters or gradient updates computed during the training. We demonstrate that the training data used by participants in collaborative learning is vulnerable to inference attacks. First, we show that an adversarial participant can infer the presence of exact data points in others' training data (i.e., membership inference). Then, we demonstrate that the adversary can infer properties that hold only for a subset of the training data and are independent of the properties that the joint model aims to capture. We evaluate the efficacy of our attacks on a variety of tasks, datasets, and learning configurations, and conclude with a discussion of possible defenses.

Motivation & Objective

• To investigate the vulnerability of collaborative learning systems to inference attacks that compromise training data privacy.
• To demonstrate that adversarial participants can infer exact data points present in others' training data through shared model parameters or gradients.
• To explore whether adversaries can infer hidden, dataset-specific properties that are not directly related to the model's primary learning objective.
• To evaluate the effectiveness of these attacks across various machine learning tasks, datasets, and collaborative learning configurations.
• To discuss potential defenses against such inference threats in collaborative learning frameworks.

Proposed method

• The authors design inference attacks that analyze shared model parameters or gradient updates exchanged during collaborative training.
• For membership inference, the attack uses statistical analysis to determine whether a specific data point was used in a participant's training set based on changes in model weights or gradients.
• For property inference, the attack identifies patterns in model updates that correlate with rare or hidden data characteristics not aligned with the main model objective.
• The attacks are evaluated on multiple datasets and learning configurations, including image classification and natural language tasks, using standard collaborative learning setups.
• The method leverages the fact that model updates reveal information about underlying training data, even when data is not directly shared.
• Experiments compare attack success rates under different model architectures, data distributions, and communication frequencies.

Experimental results

Research questions

• RQ1 Can an adversarial participant in a collaborative learning system infer whether a specific data point was used in another participant's training set?
• RQ2 To what extent can an adversary infer hidden, dataset-specific properties that are not part of the primary learning objective?
• RQ3 How effective are these inference attacks across different machine learning tasks, datasets, and collaborative learning configurations?
• RQ4 What factors influence the success rate of these inference attacks in collaborative learning systems?
• RQ5 What are the implications of these attacks for the privacy guarantees of federated and distributed learning?

Key findings

• The membership inference attack successfully identifies whether a specific data point was present in another participant's training data with high accuracy across multiple datasets and model types.
• The property inference attack can detect rare or hidden data characteristics not aligned with the model's main objective, indicating that model updates leak unintended information.
• Attack success rates remain high even under realistic collaborative learning settings, including with non-IID data distributions and varying communication intervals.
• The vulnerability is consistent across diverse tasks such as image classification and natural language processing, indicating broad applicability of the threat.
• The results demonstrate that collaborative learning systems are inherently susceptible to inference attacks due to the exposure of model parameters and gradients.
• The study reveals that current collaborative learning protocols do not adequately protect training data privacy, necessitating stronger defense mechanisms.