[Paper Review] Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks

Yige Li, Xixiang Lyu+4 more|arXiv (Cornell University)|Jan 15, 2021

Adversarial Robustness in Machine Learning53 references139 citations

TL;DR

The paper introduces Neural Attention Distillation (NAD), a distillation-based defense that uses a teacher network to guide finetuning of a backdoored student network so that attention maps align; NAD effectively erases backdoor triggers with only 5% clean data across several attacks, preserving clean accuracy.

ABSTRACT

Deep neural networks (DNNs) are known vulnerable to backdoor attacks, a training time attack that injects a trigger pattern into a small proportion of training data so as to control the model's prediction at the test time. Backdoor attacks are notably dangerous since they do not affect the model's performance on clean examples, yet can fool the model to make incorrect prediction whenever the trigger pattern appears during testing. In this paper, we propose a novel defense framework Neural Attention Distillation (NAD) to erase backdoor triggers from backdoored DNNs. NAD utilizes a teacher network to guide the finetuning of the backdoored student network on a small clean subset of data such that the intermediate-layer attention of the student network aligns with that of the teacher network. The teacher network can be obtained by an independent finetuning process on the same clean subset. We empirically show, against 6 state-of-the-art backdoor attacks, NAD can effectively erase the backdoor triggers using only 5\% clean training data without causing obvious performance degradation on clean examples. Code is available in https://github.com/bboylyg/NAD.

Motivation & Objective

• Motivate robust defense against backdoor attacks where triggers do not affect clean accuracy but hijack predictions.
• Develop a finetuning framework that uses knowledge distillation and neural attention transfer to erase backdoor triggers.
• Show that NAD works across multiple backdoor attack types with limited clean data while maintaining clean accuracy.

Proposed method

• Define an attention operator A to produce attention maps from layer activations.
• Compute NAD loss as the normalized L2 distance between teacher and student attention maps across residual groups (Equation 2).
• Train a teacher network by finetuning the backdoored model on a small clean data subset.
• Optimize a total loss L_total combining cross-entropy on the student and NAD losses (Equation 3).
• Use 5% clean data and 10 training epochs with SGD to finetune the student and form the teacher.

Experimental results

Research questions

• RQ1 Can NAD remove backdoor triggers with minimal clean data across diverse backdoor attacks?
• RQ2 How does attention-based distillation compare to standard finetuning and other erasing methods in terms of ASR and clean ACC?
• RQ3 What choices of attention representations and teacher-student configurations yield the best erasing performance?

Key findings

• NAD reduces average attack success rate (ASR) from about 99–100% to around 7.2% on average across six attacks, with minimal drop in clean accuracy (~2.7%).
• Among attack types, NAD consistently outperforms standard finetuning, fine-pruning, and mode connectivity repair in erasing backdoors.
• Attention-based distillation (especially A_sum^2) provides stronger backdoor erasure and clearer separation between backdoored and benign regions than direct feature distillation.
• NAD can converge quickly (a few epochs) and remains effective across different teacher-student architectural choices and varying amounts of clean data.