[Paper Review] Learning to Detect Malicious Clients for Robust Federated Learning

Suyi Li, Yong Cheng+3 more|arXiv (Cornell University)|Feb 1, 2020

Privacy-Preserving Technologies in Data24 references185 citations

TL;DR

This paper introduces a spectral anomaly detection framework at the server to identify and remove malicious client updates in federated learning, defending against both untargeted (Byzantine) and targeted (backdoor) attacks.

ABSTRACT

Federated learning systems are vulnerable to attacks from malicious clients. As the central server in the system cannot govern the behaviors of the clients, a rogue client may initiate an attack by sending malicious model updates to the server, so as to degrade the learning performance or enforce targeted model poisoning attacks (a.k.a. backdoor attacks). Therefore, timely detecting these malicious model updates and the underlying attackers becomes critically important. In this work, we propose a new framework for robust federated learning where the central server learns to detect and remove the malicious model updates using a powerful detection model, leading to targeted defense. We evaluate our solution in both image classification and sentiment analysis tasks with a variety of machine learning models. Experimental results show that our solution ensures robust federated learning that is resilient to both the Byzantine attacks and the targeted model poisoning attacks.

Motivation & Objective

• Motivate robust federated learning by addressing malicious client updates that can degrade or poison models.
• Propose a spectral anomaly detection framework to detect malicious updates in low-dimensional embeddings.
• Show that detecting and removing malicious updates yields robust performance across tasks and models.

Proposed method

• Embed model updates into a low-dimensional latent space using a variational autoencoder (VAE).
• Train the VAE with normal (unbiased) updates using public data, enabling reconstruction-error based anomaly detection.
• Apply dynamic thresholding at the server per round to decide which client updates to discard.
• Aggregate only the benign updates with FedAvg-style weighting based on local data sizes.
• Provide theoretical intuition via a linear model analysis of malicious update impact (Theorem 1).
• Benchmark against standard defenses (GeoMed, Krum) on image classification and sentiment tasks.

Experimental results

Research questions

• RQ1 Can spectral anomaly detection distinguish malicious from benign client updates in FL under non-IID data?
• RQ2 Does detecting and removing malicious updates improve robustness against untargeted and backdoor attacks across architectures and datasets?
• RQ3 How does the proposed method compare to existing Byzantine-robust aggregators in practice?

Key findings

• The proposed method achieves higher robustness than baselines against sign-flipping, additive noise, and backdoor attacks across MNIST, FEMNIST, and Sentiment140.
• Detection models attain high F1 scores in distinguishing benign vs malicious clients (F1: FEMNIST 1.00/0.97/0.87; MNIST 1.00/0.99/1.00; Sentiment140 1.00/1.00/0.93).
• Dynamic thresholding based on reconstruction error enables effective removal of malicious updates without manual tuning per round.
• The approach preserves convergence properties similar to FedAvg while providing targeted defense against adversaries.
• Compared to Krum, GeoMed, the proposed method offers stronger protection against targeted backdoor attacks on several datasets.