[Paper Review] Data Poisoning Attacks Against Federated Learning Systems

Vale Tolpegin, Stacey Truex+2 more|arXiv (Cornell University)|Jul 16, 2020

Adversarial Robustness in Machine LearningComputer Science56 references51 citations

TL;DR

The paper demonstrates targeted data poisoning in federated learning via label flipping, showing significant drops in global accuracy and source-class recall, and introduces a PCA-based defense to identify malicious updates.

ABSTRACT

Federated learning (FL) is an emerging paradigm for distributed training of large-scale deep neural networks in which participants' data remains on their own devices with only model updates being shared with a central server. However, the distributed nature of FL gives rise to new threats caused by potentially malicious participants. In this paper, we study targeted data poisoning attacks against FL systems in which a malicious subset of the participants aim to poison the global model by sending model updates derived from mislabeled data. We first demonstrate that such data poisoning attacks can cause substantial drops in classification accuracy and recall, even with a small percentage of malicious participants. We additionally show that the attacks can be targeted, i.e., they have a large negative impact only on classes that are under attack. We also study attack longevity in early/late round training, the impact of malicious participant availability, and the relationships between the two. Finally, we propose a defense strategy that can help identify malicious participants in FL to circumvent poisoning attacks, and demonstrate its effectiveness.

Motivation & Objective

• Motivate the study of privacy-preserving distributed learning and its vulnerability to malicious participants.
• Characterize targeted data poisoning attacks in federated learning using label flipping.
• Evaluate attack impact under varying attacker presence, timing, and availability.
• Propose and empirically validate a defense strategy to identify malicious participants in FL.

Proposed method

• Formalize a federated learning setup with an honest aggregator and multiple participants holding local data.
• Use label flipping (src -> target) as the poisoning strategy for malicious participants.
• Evaluate impact on CIFAR-10 and Fashion-MNIST using CNN architectures under varying malicious-participant rates.
• Analyze attack timing (early vs late poisoning) and malicious-participant availability on attack efficacy.
• Propose a defense where the aggregator detects malicious updates by extracting update components, applying PCA for dimensionality reduction, and identifying outliers.

Experimental results

Research questions

• RQ1 How effective are targeted label flipping attacks in federated learning at reducing global model utility?
• RQ2 Do label flipping attacks preferentially degrade specific source/target classes versus remaining classes?
• RQ3 How do attack timing and malicious-participant availability affect attack impact in FL?
• RQ4 Can a PCA-based defense reliably distinguish malicious updates from honest ones to mitigate poisoning?

Key findings

• Label flipping attacks can substantially reduce global model accuracy and source-class recall even with a minority of malicious participants.
• Attacks tend to be targeted, causing large declines in source and target class recalls while leaving other classes relatively preserved.
• Late-round poisoning and higher malicious-participant availability increase attack effectiveness, while the model can recover after poisoning ends in many scenarios.
• A defense using PCA on processed update components can separate malicious updates from honest ones, enabling the aggregator to identify and block attackers.
• Attack impact is more pronounced on the targeted source class recall than on other metrics, indicating a focused poisoning effect.