[Paper Review] Byzantine-Robust Decentralized Federated Learning

Minghong Fang, Zifan Zhang+6 more|arXiv (Cornell University)|Jun 14, 2024

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes BALANCE, a Byzantine-robust decentralized federated learning algorithm that uses each client’s local model as a similarity reference to detect and reject malicious updates. It achieves convergence rates matching state-of-the-art methods in Byzantine-free settings and outperforms existing DFL defenses under both LIE and Dissensus poisoning attacks.

ABSTRACT

Federated learning (FL) enables multiple clients to collaboratively train machine learning models without revealing their private training data. In conventional FL, the system follows the server-assisted architecture (server-assisted FL), where the training process is coordinated by a central server. However, the server-assisted FL framework suffers from poor scalability due to a communication bottleneck at the server, and trust dependency issues. To address challenges, decentralized federated learning (DFL) architecture has been proposed to allow clients to train models collaboratively in a serverless and peer-to-peer manner. However, due to its fully decentralized nature, DFL is highly vulnerable to poisoning attacks, where malicious clients could manipulate the system by sending carefully-crafted local models to their neighboring clients. To date, only a limited number of Byzantine-robust DFL methods have been proposed, most of which are either communication-inefficient or remain vulnerable to advanced poisoning attacks. In this paper, we propose a new algorithm called BALANCE (Byzantine-robust averaging through local similarity in decentralization) to defend against poisoning attacks in DFL. In BALANCE, each client leverages its own local model as a similarity reference to determine if the received model is malicious or benign. We establish the theoretical convergence guarantee for BALANCE under poisoning attacks in both strongly convex and non-convex settings. Furthermore, the convergence rate of BALANCE under poisoning attacks matches those of the state-of-the-art counterparts in Byzantine-free settings. Extensive experiments also demonstrate that BALANCE outperforms existing DFL methods and effectively defends against poisoning attacks.

Motivation & Objective

• Address the vulnerability of decentralized federated learning (DFL) to poisoning attacks, particularly in peer-to-peer, serverless architectures.
• Overcome the limitations of adapting server-assisted Byzantine-robust defenses to DFL, which often fail due to architectural mismatch.
• Design a communication-efficient, decentralized defense mechanism that does not rely on a central server or global model aggregation.
• Establish theoretical convergence guarantees for the proposed method under both strongly convex and non-convex settings in the presence of Byzantine clients.
• Demonstrate superior robustness against advanced poisoning attacks such as LIE and Dissensus, which are tailored for decentralized environments.

Proposed method

• BALANCE employs a local similarity-based filtering mechanism where each client uses its own current local model as a reference to evaluate the trustworthiness of received models from neighbors.
• A client accepts a neighbor’s model only if the L2 distance between its own model and the received model is within a threshold proportional to its own model norm.
• The method dynamically adjusts the acceptance threshold using an exponential decay function to reduce false positives in later training stages.
• Two variants are evaluated: Variant I uses a fixed threshold based on relative distance, while Variant II introduces a median-based deviation threshold to improve robustness.
• The algorithm ensures convergence by maintaining model consistency across clients through similarity filtering, even when up to a constant fraction of neighbors are malicious.
• Theoretical analysis proves convergence under both strongly convex and non-convex objectives, with convergence rates matching those of non-robust baselines in Byzantine-free settings.

Experimental results

Research questions

• RQ1 Can a decentralized federated learning system achieve Byzantine robustness without relying on a central server or global model aggregation?
• RQ2 How can local model similarity be leveraged to detect malicious updates in a fully decentralized peer-to-peer setting?
• RQ3 Does the proposed BALANCE method maintain convergence rates comparable to non-robust baselines under poisoning attacks?
• RQ4 How effective is BALANCE against advanced, DFL-specific poisoning attacks such as the Dissensus attack and LIE attack?
• RQ5 Can the thresholding mechanism in BALANCE be tuned to balance robustness and model accuracy across different training phases?

Key findings

• BALANCE achieves convergence rates under poisoning attacks that match those of state-of-the-art Byzantine-free methods in both strongly convex and non-convex settings.
• Under the Dissensus attack, BALANCE maintains a Max.TER of 0.11, significantly outperforming the Median method (0.87) and Trim-mean (0.72), demonstrating strong resilience.
• The LIE attack has limited impact on BALANCE, with Max.TER remaining at 0.11, indicating robustness against generic poisoning strategies.
• Variant I of BALANCE performs comparably to the full BALANCE method, while Variant II underperforms due to over-rejection of benign models.
• The Max.TER for the Median method increases from 0.38 (attacks only in second half of training) to 0.55 (first half) and 0.63 (entire training), showing that attack timing affects performance.
• The exponential decay in the threshold parameter reduces the risk of accepting malicious models in later training stages, enhancing long-term robustness.