[Paper Review] Eavesdrop the Composition Proportion of Training Labels in Federated Learning

Lixu Wang, Shichao Xu+2 more|arXiv (Cornell University)|Oct 14, 2019

Privacy-Preserving Technologies in Data52 references34 citations

TL;DR

The paper presents three inference attacks—Class Sniffing, Quantity Inference, and Whole Determination—that reveal the composition proportion of training labels in federated learning without observing individual client updates, even under secure aggregation or differential privacy.

ABSTRACT

Federated learning (FL) has recently emerged as a new form of collaborative machine learning, where a common model can be learned while keeping all the training data on local devices. Although it is designed for enhancing the data privacy, we demonstrated in this paper a new direction in inference attacks in the context of FL, where valuable information about training data can be obtained by adversaries with very limited power. In particular, we proposed three new types of attacks to exploit this vulnerability. The first type of attack, Class Sniffing, can detect whether a certain label appears in training. The other two types of attacks can determine the quantity of each label, i.e., Quantity Inference attack determines the composition proportion of the training label owned by the selected clients in a single round, while Whole Determination attack determines that of the whole training process. We evaluated our attacks on a variety of tasks and datasets with different settings, and the corresponding results showed that our attacks work well generally. Finally, we analyzed the impact of major hyper-parameters to our attacks and discussed possible defenses.

Motivation & Objective

• Introduce a new privacy vulnerability surface in federated learning: inferring the quantity composition of training labels.
• Propose three attacks (Class Sniffing, Quantity Inference, Whole Determination) that do not rely on observing individual gradient updates.
• Demonstrate effectiveness across varied tasks/datasets and discuss hyper-parameter impacts and defenses.

Proposed method

• Class Sniffing: infer whether a specific label appears in the training round by analyzing updates to input connections of output neurons.
• Quantity Inference: estimate how many clients own a given label by comparing positive/negative weight update magnitudes and removing offset effects.
• Whole Determination: assess the label composition proportion across the entire training process using ratio metrics and clustering of derived features.

Experimental results

Research questions

• RQ1 Can an attacker determine the presence of a specific label in a single FL training round without observing individual updates?
• RQ2 Can an attacker infer the quantity composition (how many clients own each label) in a single round and across the full training process without access to plain updates?
• RQ3 Can the composition proportions of training labels be estimated robustly against aggregation protections like secure aggregation or differential privacy?

Key findings

• Three new label-quantity inference attacks achieve high success rates in detecting label presence and estimating label counts.
• Attacks remain effective under secure aggregation and differential privacy settings since they rely on global model updates and auxiliary data.
• Quantitative techniques enable single-round and full-training-round leakage of label composition, illustrating a new dimension of privacy risk in FL.