[Paper Review] Defending against Backdoor Attack on Deep Neural Networks

Hao Cheng, Kaidi Xu+4 more|arXiv (Cornell University)|Feb 26, 2020

Adversarial Robustness in Machine Learning23 references37 citations

TL;DR

The paper analyzes how backdoor attacks alter DNN activation patterns and proposes an L-infinity based neuron pruning defense to reduce attack success while preserving accuracy on clean data.

ABSTRACT

Although deep neural networks (DNNs) have achieved a great success in various computer vision tasks, it is recently found that they are vulnerable to adversarial attacks. In this paper, we focus on the so-called extit{backdoor attack}, which injects a backdoor trigger to a small portion of training data (also known as data poisoning) such that the trained DNN induces misclassification while facing examples with this trigger. To be specific, we carefully study the effect of both real and synthetic backdoor attacks on the internal response of vanilla and backdoored DNNs through the lens of Gard-CAM. Moreover, we show that the backdoor attack induces a significant bias in neuron activation in terms of the $\ell_\infty$ norm of an activation map compared to its $\ell_1$ and $\ell_2$ norm. Spurred by our results, we propose the extit{$\ell_\infty$-based neuron pruning} to remove the backdoor from the backdoored DNN. Experiments show that our method could effectively decrease the attack success rate, and also hold a high classification accuracy for clean images.

Motivation & Objective

• Characterize vanilla and backdoored DNN responses to backdoor triggers using Grad-CAM.
• Quantitatively analyze neuron activation patterns via p-norms to identify robust defense criteria.
• Propose and evaluate an L-infinity norm based neuron pruning method to mitigate backdoor effects.
• Demonstrate trade-offs between attack suppression and clean accuracy on a traffic sign dataset.

Proposed method

• Use Grad-CAM to visualize discriminative regions for vanilla and backdoored DNNs under clean and triggered inputs.
• Compute neuron activation maps of the final convolutional layer and compare L1, L2, and L-infinity norms across input conditions.
• Synthesize backdoor triggers via trigger reverse engineering and evaluate four input conditions (clean, clean+ori, clean+syn, clean+ori+syn).
• Identify that L-infinity norm shows the strongest distinction between clean and triggered cases.
• Apply pruning to neurons with high L-infinity activation to suppress trigger-induced predictions.
• Evaluate pruning on German Traffic Sign Recognition Benchmark using AlexNet, reporting accuracy and attack success rate.

Experimental results

Research questions

• RQ1 Can Grad-CAM reveal differences in how vanilla and backdoored DNNs respond to backdoor triggers?
• RQ2 Which activation-norm metric best discriminates between clean and triggered activations?
• RQ3 Does L-infinity based neuron pruning effectively reduce backdoor attack success while preserving clean accuracy?
• RQ4 What is the trade-off between defense strength and model performance on clean data?

Key findings

• Backdoored DNNs show discriminative activation focused on the trigger region when the trigger is present.
• L-infinity norm of final-layer activations exhibits the largest difference between clean and triggered inputs compared to L1 and L2.
• L-infinity based pruning reduces attack success rates from 81.61% to as low as 42.99–48.42% with modest clean accuracy loss.
• Optimal pruning thresholds achieve a balance where clean accuracy falls minimally while attack success is substantially reduced.