[Paper Review] The Hidden Vulnerability of Distributed Learning in Byzantium

El Mahdi El Mhamdi, Rachid Guerraoui+1 more|arXiv (Cornell University)|Feb 22, 2018

Stochastic Gradient Optimization Techniques20 references473 citations

TL;DR

The paper shows that Byzantine-resilient aggregation in distributed SGD can still guide training to ineffective models in high dimensions, and it introduces Bulyan to dramatically reduce attacker leeway to O(1/√d), with empirical validation on MNIST and CIFAR-10.

ABSTRACT

While machine learning is going through an era of celebrated success, concerns have been raised about the vulnerability of its backbone: stochastic gradient descent (SGD). Recent approaches have been proposed to ensure the robustness of distributed SGD against adversarial (Byzantine) workers sending poisoned gradients during the training phase. Some of these approaches have been proven Byzantine-resilient: they ensure the convergence of SGD despite the presence of a minority of adversarial workers. We show in this paper that convergence is not enough. In high dimension $d \gg 1$, an adver\-sary can build on the loss function's non-convexity to make SGD converge to ineffective models. More precisely, we bring to light that existing Byzantine-resilient schemes leave a margin of poisoning of $Ω\left(f(d) ight)$, where $f(d)$ increases at least like $\sqrt{d~}$. Based on this leeway, we build a simple attack, and experimentally show its strong to utmost effectivity on CIFAR-10 and MNIST. We introduce Bulyan, and prove it significantly reduces the attackers leeway to a narrow $O( \frac{1}{\sqrt{d~}})$ bound. We empirically show that Bulyan does not suffer the fragility of existing aggregation rules and, at a reasonable cost in terms of required batch size, achieves convergence as if only non-Byzantine gradients had been used to update the model.

Motivation & Objective

• Motivate the study of robustness in distributed SGD under Byzantine faults in high-dimensional, non-convex settings.
• Assess whether convergence guarantees of Byzantine-resilient aggregation rules are sufficient for neural networks.
• Demonstrate the existence of an attack exploiting the curse of dimensionality against ℓp-based GARs.
• Propose a general enhancement (Bulyan) to tighten Byzantine-leeway and prove convergence.
• Empirically validate the approach on MNIST and CIFAR-10 and analyze the computational trade-offs.

Proposed method

• Describe the distributed SGD model with a master and f Byzantine workers under a gradient-aggregation rule (GAR).
• Characterize a simple attack that leverages high-dimensional loss landscapes to push the aggregated gradient toward suboptimal regions.
• Introduce Bulyan, a two-step enhancement that uses an underlying Byzantine-resilient rule A to select a set of gradients, then aggregates coordinates via β-nearest values to the coordinate medians.
• Prove theoretical bounds: (i) Bulyan reduces Byzantine leeway to O(1/√d) per coordinate, (ii) Bulyan preserves convergence under the same α, f bounds as A.
• Provide complexity analysis showing Bulyan’s cost is O((n−2f)C + dn) per epoch, and in practice O(n^2 d) for GeoMed/Krum variants.
• Empirically compare Bulyan against Brute, Krum, GeoMed on MNIST and CIFAR-10, and study convergence speed and robustness.

Experimental results

Research questions

• RQ1 Can Byzantine-resilient gradient aggregation guarantee convergence in high-dimensional, non-convex neural networks?
• RQ2 How much adversarial leeway do existing GARs have in large-scale, non-convex settings?
• RQ3 Can we design an augmentation to GARs that narrows the attacker's influence without sacrificing convergence?
• RQ4 Does the proposed Bulyan method restore robust convergence and how does it impact training speed in practice?

Key findings

• Convergence guarantees of existing Byzantine-resilient GARs can still yield ineffectual models when facing a single Byzantine worker in high dimensions.
• There exists an Ω(f(d)) poisoning margin for ℓp-based GARs, growing at least as √d, enabling effective attacks.
• A generic enhancement, Bulyan(A), tightly constrains the attacker's coordinate-wise influence to O(σ/√d) and retains convergence.
• Empirical results on MNIST and CIFAR-10 show Bulyan with A (e.g., Krum) achieves accuracy comparable to non-Byzantine averaging and resists the proposed attack.
• In non-Byzantine scenarios, Bulyan incurs modest convergence-speed costs, which peak with small batch sizes and can be minimized with reasonable batch sizing.
• Bulyan preserves the convergence guarantees (almost surely) under the (α, f)-Byzantine-resilience framework.