[Paper Review] BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers

Xin Fan, Yue Wang+2 more|arXiv (Cornell University)|Oct 18, 2021

Privacy-Preserving Technologies in Data51 references4 citations

TL;DR

This paper proposes BEV-SGD, a novel Byzantine-resilient power control scheme for analog aggregation-based federated learning over the air (FLOA), where all workers transmit at maximum power to enhance robustness. By eliminating channel inversion and leveraging a best-effort voting mechanism, BEV-SGD achieves superior convergence rates under worst-case Byzantine attacks compared to conventional channel-inversion methods.

ABSTRACT

As a promising distributed learning technology, analog aggregation based federated learning over the air (FLOA) provides high communication efficiency and privacy provisioning under the edge computing paradigm. When all edge devices (workers) simultaneously upload their local updates to the parameter server (PS) through commonly shared time-frequency resources, the PS obtains the averaged update only rather than the individual local ones. While such a concurrent transmission and aggregation scheme reduces the latency and communication costs, it unfortunately renders FLOA vulnerable to Byzantine attacks. Aiming at Byzantine-resilient FLOA, this paper starts from analyzing the channel inversion (CI) mechanism that is widely used for power control in FLOA. Our theoretical analysis indicates that although CI can achieve good learning performance in the benign scenarios, it fails to work well with limited defensive capability against Byzantine attacks. Then, we propose a novel scheme called the best effort voting (BEV) power control policy that is integrated with stochastic gradient descent (SGD). Our BEV-SGD enhances the robustness of FLOA to Byzantine attacks, by allowing all the workers to send their local updates at their maximum transmit power. Under worst-case attacks, we derive the expected convergence rates of FLOA with CI and BEV power control policies, respectively. The rate comparison reveals that our BEV-SGD outperforms its counterpart with CI in terms of better convergence behavior, which is verified by experimental simulations.

Motivation & Objective

• Address the vulnerability of analog aggregation-based federated learning over the air (FLOA) to Byzantine attacks, where malicious workers can disrupt model convergence.
• Identify the limitations of channel inversion (CI) power control in defending against Byzantine attacks despite its strong performance in benign scenarios.
• Design a new power control policy—Best Effort Voting (BEV)—that enhances robustness by allowing all workers to transmit at maximum power, avoiding reliance on individual gradient knowledge.
• Theoretically analyze and compare convergence rates of FLOA under CI and BEV power control policies under worst-case Byzantine attacks.
• Demonstrate through analysis and simulations that BEV-SGD achieves better convergence behavior and resilience than CI-based FLOA in adversarial settings.

Proposed method

• Propose the Best Effort Voting (BEV) power control policy, where all workers, including Byzantine ones, transmit at their maximum power to maximize signal strength and voting weight.
• Integrate BEV with stochastic gradient descent (SGD) to form BEV-SGD, enabling robust aggregation without requiring individual gradient values.
• Theoretical analysis derives expected convergence rates for FLOA under both CI and BEV policies, using bounds on gradient norms, noise variances, and attack magnitudes.
• Define key convergence metrics: $ \omega_{BEV} $, the effective signal gain difference between honest and Byzantine workers, and $ \Omega_{BEV} $, the total variance term in the error bound.
• Establish convergence conditions by ensuring $ \alpha \omega_{BEV} - \frac{\alpha^2 L}{2} \Omega_{BEV} > 0 $, guaranteeing monotonic decrease in objective function in expectation.
• Perform telescoping sum over T iterations and derive a $ O(1/\sqrt{T}) $ convergence rate, showing BEV-SGD's robustness under adversarial noise and channel distortion.

Experimental results

Research questions

• RQ1 How does channel inversion (CI) power control perform under Byzantine attacks in FLOA, despite its strong performance in benign environments?
• RQ2 Can a max-power transmission strategy with voting-based aggregation improve Byzantine resilience in FLOA, where individual gradients are not accessible?
• RQ3 What is the theoretical convergence rate of FLOA under the proposed BEV-SGD scheme under worst-case Byzantine attacks?
• RQ4 How does the convergence behavior of BEV-SGD compare to CI-based FLOA in terms of expected objective decrease and error bounds?
• RQ5 What are the key system parameters (e.g., power, channel gains, noise) that influence the convergence rate and robustness of BEV-SGD?

Key findings

• BEV-SGD achieves a theoretical convergence rate of $ O(1/\sqrt{T}) $, with the error bound scaling as $ \frac{\frac{2L\Omega_{BEV}}{\bar{\alpha}\omega_{BEV}^{2}}(F(\mathbf{w}_0)-F(\mathbf{w}^*)) + \bar{\alpha}\left(\delta^{2} + \frac{1}{\Omega_{BEV}}\epsilon^{2}z^{2}\right)}{\sqrt{T}} $, demonstrating asymptotic convergence.
• The convergence condition $ \alpha \omega_{BEV} - \frac{\alpha^2 L}{2} \Omega_{BEV} > 0 $ ensures monotonic decrease in the expected objective function, validating convergence in mean.
• BEV-SGD outperforms CI-based FLOA in convergence behavior under worst-case Byzantine attacks, as shown by the derived rate comparison.
• The proposed BEV power control policy eliminates dependency on individual gradient knowledge, making it suitable for FLOA’s analog superposition model.
• Theoretical analysis confirms that BEV-SGD maintains robustness even when Byzantine workers inject large-scale perturbations, due to the voting mechanism favoring honest gradients.
• Simulations verify that BEV-SGD maintains stable learning performance under Byzantine attacks, while CI-based schemes degrade significantly.