[Paper Review] Over-The-Air Federated Learning under Byzantine Attacks

Houssem Sifaou, Geoffrey Ye Li|arXiv (Cornell University)|May 5, 2022

Privacy-Preserving Technologies in Data4 citations

TL;DR

This paper proposes ROTAF, a robust over-the-air federated learning framework that groups clients and uses geometric median aggregation to defend against Byzantine attacks during AirComp-based model updates. The method ensures linear convergence to a neighborhood of the optimal solution even when less than half the groups are compromised, with convergence error bounded by noise variance and attacker count.

ABSTRACT

Federated learning (FL) is a promising solution to enable many AI applications, where sensitive datasets from distributed clients are needed for collaboratively training a global model. FL allows the clients to participate in the training phase, governed by a central server, without sharing their local data. One of the main challenges of FL is the communication overhead, where the model updates of the participating clients are sent to the central server at each global training round. Over-the-air computation (AirComp) has been recently proposed to alleviate the communication bottleneck where the model updates are sent simultaneously over the multiple-access channel. However, simple averaging of the model updates via AirComp makes the learning process vulnerable to random or intended modifications of the local model updates of some Byzantine clients. In this paper, we propose a transmission and aggregation framework to reduce the effect of such attacks while preserving the benefits of AirComp for FL. For the proposed robust approach, the central server divides the participating clients randomly into groups and allocates a transmission time slot for each group. The updates of the different groups are then aggregated using a robust aggregation technique. We extend our approach to handle the case of non-i.i.d. local data, where a resampling step is added before robust aggregation. We analyze the convergence of the proposed approach for both cases of i.i.d. and non-i.i.d. data and demonstrate that the proposed algorithm converges at a linear rate to a neighborhood of the optimal solution. Experiments on real datasets are provided to confirm the robustness of the proposed approach.

Motivation & Objective

• To address the vulnerability of over-the-air federated learning (OTA-FL) to Byzantine attacks, where malicious clients manipulate model updates.
• To preserve the communication efficiency of AirComp while ensuring robustness against adversarial clients.
• To enable convergence under non-i.i.d. data distributions via a resampling step before robust aggregation.
• To theoretically analyze convergence for both i.i.d. and non-i.i.d. data, proving linear convergence to a neighborhood of the optimal solution.
• To validate the robustness and effectiveness of the proposed framework through experiments on real datasets.

Proposed method

• The central server divides participating clients into groups and assigns each group a dedicated time slot for simultaneous over-the-air transmission.
• Model updates from different groups are aggregated using geometric median aggregation to resist Byzantine manipulations.
• For non-i.i.d. data, a resampling step is applied before robust aggregation to improve convergence stability.
• The framework uses a resampling mechanism with s-replacement to generate stable update estimates, reducing variance from malicious inputs.
• Theoretical analysis relies on bounding the error in geometric median estimation using probabilistic concentration and resampling techniques.
• Convergence is proven under standard assumptions on loss functions, showing linear convergence rate with asymptotic error dependent on noise and attacker count.

Experimental results

Research questions

• RQ1 Can over-the-air federated learning be made robust to Byzantine attacks without sacrificing communication efficiency?
• RQ2 How does group-based transmission and geometric median aggregation improve resilience compared to simple averaging in AirComp?
• RQ3 What is the theoretical convergence behavior of the proposed method under i.i.d. and non-i.i.d. data distributions?
• RQ4 How does the resampling step affect robustness and convergence in the presence of heterogeneous data?
• RQ5 What is the upper bound on the asymptotic learning error in the presence of Byzantine attackers?

Key findings

• The proposed ROTAF framework achieves linear convergence to a neighborhood of the optimal solution under both i.i.d. and non-i.i.d. data settings.
• When the number of Byzantine groups is less than half the total number of groups, the algorithm remains robust and converges reliably.
• The asymptotic learning error is bounded by a term proportional to the noise variance and the number of Byzantine attackers.
• The resampling step significantly improves convergence stability under non-i.i.d. data, reducing the impact of data heterogeneity.
• Numerical experiments confirm that ROTAF outperforms simple averaging OTA-FL in robustness against Byzantine attacks.
• Theoretical analysis shows that the error bound scales with the inverse of the group size and depends on the geometric median estimation error, which is controlled via resampling.