[Paper Review] Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks

Sanghyun Hong, Pietro Frigo+3 more|arXiv (Cornell University)|Jun 3, 2019

Adversarial Robustness in Machine Learning59 references103 citations

TL;DR

The paper analyzes how single-bit flips in DNN parameters, induced by hardware fault attacks like Rowhammer, can cause severe, non-graceful accuracy degradation across 19 models and multiple datasets, and proposes mitigations.

ABSTRACT

Deep neural networks (DNNs) have been shown to tolerate "brain damage": cumulative changes to the network's parameters (e.g., pruning, numerical perturbations) typically result in a graceful degradation of classification accuracy. However, the limits of this natural resilience are not well understood in the presence of small adversarial changes to the DNN parameters' underlying memory representation, such as bit-flips that may be induced by hardware fault attacks. We study the effects of bitwise corruptions on 19 DNN models---six architectures on three image classification tasks---and we show that most models have at least one parameter that, after a specific bit-flip in their bitwise representation, causes an accuracy loss of over 90%. We employ simple heuristics to efficiently identify the parameters likely to be vulnerable. We estimate that 40-50% of the parameters in a model might lead to an accuracy drop greater than 10% when individually subjected to such single-bit perturbations. To demonstrate how an adversary could take advantage of this vulnerability, we study the impact of an exemplary hardware fault attack, Rowhammer, on DNNs. Specifically, we show that a Rowhammer enabled attacker co-located in the same physical machine can inflict significant accuracy drops (up to 99%) even with single bit-flip corruptions and no knowledge of the model. Our results expose the limits of DNNs' resilience against parameter perturbations induced by real-world fault attacks. We conclude by discussing possible mitigations and future research directions towards fault attack-resilient DNNs.

Motivation & Objective

• Assess the vulnerability of DNN parameters to single-bit flips under hardware fault attacks.
• Characterize how bit position, flip direction, parameter sign, and architecture affect vulnerability.
• Evaluate practical attack scenarios (Rowhammer) in realistic MLaaS settings.
• Identify potential mitigations to improve fault-resilience of DNNs.

Proposed method

• Systematically flip each bit in every parameter of a model and measure misclassification rate on a validation set.
• Analyze 19 DNN models across MNIST, CIFAR10, and ImageNet to determine vulnerable parameters (RAD>0.1).
• Use speed-up heuristics (Sampled Validation, Specific Bits, Sampled Parameters) to handle large models.
• Simulate Rowhammer attacks in a co-located MLaaS scenario to assess practical impact.
• Evaluate mitigations such as activation magnitude restriction (ReLU6) and quantization/binarization of weights.

Experimental results

Research questions

• RQ1 How vulnerable are DNN parameters to single-bit flips across different architectures and datasets?
• RQ2 Which bit positions, flip directions, and parameter signs contribute most to indiscriminate damage (RAD>0.1)?
• RQ3 Can practical hardware fault attacks like Rowhammer cause large accuracy drops in co-located MLaaS settings?
• RQ4 Do common training techniques (dropout, batch normalization) mitigate single-bit vulnerability?
• RQ5 What mitigations effectively reduce vulnerability without sacrificing too much accuracy?

Key findings

• On average, ~50% of parameters are vulnerable to single-bit flips causing RAD>0.1 across models.
• Some parameters can cause >90% accuracy loss with specific bit flips, implying graceless degradation.
• Vulnerability mainly arises from large value spikes, with exponent bits (especially the 31st) being most impactful.
• Positive parameters are more vulnerable under ReLU, but other activations that allow negative outputs increase risk from negative parameters as well.
• Increasing layer width scales vulnerable parameter counts, while dropout/batch normalization provide limited protection.
• Rowhammer-enabled attackers can cause up to 99% accuracy drops in practical scenarios without model knowledge.