[Paper Review] Threats to Federated Learning: A Survey

Lingjuan Lyu, Han Yu+1 more|arXiv (Cornell University)|Mar 4, 2020

Privacy-Preserving Technologies in Data43 references234 citations

TL;DR

This survey reviews threat models in federated learning, focusing on poisoning and inference attacks by insiders, and discusses defense directions and research gaps.

ABSTRACT

With the emergence of data silos and popular privacy awareness, the traditional centralized approach of training artificial intelligence (AI) models is facing strong challenges. Federated learning (FL) has recently emerged as a promising solution under this new reality. Existing FL protocol design has been shown to exhibit vulnerabilities which can be exploited by adversaries both within and without the system to compromise data privacy. It is thus of paramount importance to make FL system designers to be aware of the implications of future FL algorithm design on privacy-preservation. Currently, there is no survey on this topic. In this paper, we bridge this important gap in FL literature. By providing a concise introduction to the concept of FL, and a unique taxonomy covering threat models and two major attacks on FL: 1) poisoning attacks and 2) inference attacks, this paper provides an accessible review of this important topic. We highlight the intuitions, key techniques as well as fundamental assumptions adopted by various attacks, and discuss promising future research directions towards more robust privacy preservation in FL.

Motivation & Objective

• Introduce federated learning and its privacy implications in the presence of data silos and privacy concerns.
• Classify FL threat models and identify insider attacks as the primary focus.
• Survey poisoning and inference attacks, including their intuitions, techniques, and assumptions.
• Discuss promising future research directions for robust privacy-preserving FL solutions.

Proposed method

• Provide a taxonomy of FL types (HFL, VFL, FTL) and participant roles (H2B, H2C) to frame threats.
• Summarize data and model poisoning attacks and their effectiveness under different FL settings.
• Describe inference attacks on gradients, including class representatives, membership, properties, and training inputs/labels leakage.
• Discuss defense mechanisms and limitations, including differential privacy and adversarial training, and outline open research directions.

Experimental results

Research questions

• RQ1 What are the main insider threat models in federated learning and how do they differ from outsider threats?
• RQ2 What poisoning and inference attack techniques threaten FL, and under which FL scenarios (H2B, H2C, VFL, FTL) are they most effective?
• RQ3 What defenses exist against FL threats, and what are their limitations in real-world, non-IID, and heterogeneous settings?
• RQ4 What future research directions can enhance robustness and privacy in federated learning?
• RQ5 How do different FL architectures and deployment scenarios influence attack surfaces and defense strategies?

Key findings

• Insider attacks on FL are typically stronger and more concerning than outsider attacks due to greater access to system state.
• Data poisoning and model poisoning can degrade or hijack FL models, with model poisoning capable of introducing stealthy backdoors.
• Inference attacks can leak private information from gradients, including class representatives, membership, properties, and training inputs/labels, especially in training-phase attacks.
• Advanced attacks like Deep Leakage from Gradients can recover training inputs and labels with few iterations, highlighting gradient leakage risks.
• Defenses such as differential privacy and adversarial training face trade-offs between privacy, accuracy, and practicality, particularly in non-IID and low-participant scenarios.