[论文解读] Mitigating Sybil Attacks on Differential Privacy based Federated Learning

Yupeng Jiang, Yong Li+2 more|arXiv (Cornell University)|Oct 20, 2020

Privacy-Preserving Technologies in Data参考文献 32被引用 6

一句话总结

本文提出了首个基于差分隐私的联邦学习中的Sybil攻击，其中恶意客户端通过调整噪声水平来操纵本地模型更新，从而降低全局模型的收敛性。该文提出一种防御机制，通过检测并过滤高损失客户端的更新，有效恢复模型收敛性和准确性，即使在攻击下也能保持良好表现。

ABSTRACT

In federated learning, machine learning and deep learning models are trained globally on distributed devices. The state-of-the-art privacy-preserving technique in the context of federated learning is user-level differential privacy. However, such a mechanism is vulnerable to some specific model poisoning attacks such as Sybil attacks. A malicious adversary could create multiple fake clients or collude compromised devices in Sybil attacks to mount direct model updates manipulation. Recent works on novel defense against model poisoning attacks are difficult to detect Sybil attacks when differential privacy is utilized, as it masks clients' model updates with perturbation. In this work, we implement the first Sybil attacks on differential privacy based federated learning architectures and show their impacts on model convergence. We randomly compromise some clients by manipulating different noise levels reflected by the local privacy budget epsilon of differential privacy on the local model updates of these Sybil clients such that the global model convergence rates decrease or even leads to divergence. We apply our attacks to two recent aggregation defense mechanisms, called Krum and Trimmed Mean. Our evaluation results on the MNIST and CIFAR-10 datasets show that our attacks effectively slow down the convergence of the global models. We then propose a method to keep monitoring the average loss of all participants in each round for convergence anomaly detection and defend our Sybil attacks based on the prediction cost reported from each client. Our empirical study demonstrates that our defense approach effectively mitigates the impact of our Sybil attacks on model convergence.

研究动机与目标

• 探究基于差分隐私的联邦学习对Sybil攻击的脆弱性。
• 展示恶意客户端如何通过与本地隐私预算ε相关的噪声水平操纵本地更新，从而降低全局模型的收敛性。
• 设计一种防御机制，检测并排除高损失客户端的更新，以恢复模型收敛性。
• 在MNIST和CIFAR-10数据集上，评估该防御机制在不同聚合规则（Krum、Trimmed Mean、FedAvg）下对Sybil攻击的有效性。

提出的方法

• 通过基于本地隐私预算ε调整噪声水平，为Sybil客户端生成本地模型更新，以诱导高预测损失。
• 监控每轮训练中各客户端报告的平均预测损失，以检测异常情况。
• 识别并排除在全局模型上引发异常高损失值的客户端更新。
• 通过在聚合前过滤掉此类高损失更新，实施防御机制。
• 采用Krum、Trimmed Mean和FedAvg作为聚合规则，评估系统的鲁棒性。
• 使用高斯噪声注入作为基线攻击，用于对比分析。

实验结果

研究问题

• RQ1 Sybil攻击能否有效降低基于差分隐私的联邦学习模型的收敛性？
• RQ2 所提出的防御机制如何检测并缓解基于差分隐私的联邦学习中的Sybil攻击？
• RQ3 在应用差分隐私的情况下，鲁棒聚合规则（Krum、Trimmed Mean）在多大程度上能抵抗所提出的Sybil攻击？
• RQ4 在Sybil攻击下，该防御能否将模型准确率恢复至接近基线水平？
• RQ5 攻击与防御在不同模型架构（CNN、MLP）和数据集（MNIST、CIFAR-10）上的表现如何？

主要发现

• 在MNIST数据集的CNN模型上，该Sybil攻击使Krum的错误率上升至14%，Trimmed Mean的错误率上升至85%，而无攻击时仅为3%。
• 在MLP模型上，使用Trimmed Mean时错误率升至65%，而防御机制可将准确率恢复至59%（基线为59%）。
• 该攻击导致FedAvg和Trimmed Mean的收敛显著变慢或出现发散，尤其在高噪声水平下更为明显。
• 防御机制成功将CNN模型的错误率恢复至3%，MLP模型的错误率恢复至59%，与无攻击基线完全一致。
• Krum和Trimmed Mean对所提出的攻击无效，而所提出的防御机制则保持了高准确率。
• 如FedAvg、Krum和Trimmed Mean的收敛曲线所示，攻击显著延缓或破坏了模型收敛。