[Paper Review] Backdoor Scanning for Deep Neural Networks through K-Arm Optimization

Guangyu Shen, Yingqi Liu+6 more|arXiv (Cornell University)|Feb 9, 2021

Adversarial Robustness in Machine Learning59 references29 citations

TL;DR

The paper introduces a K-Arm backdoor scanner that uses a multi-armed bandit strategy to efficiently identify backdoors in DNNs, achieving top performance on TrojAI with faster scanning than state-of-the-art methods.

ABSTRACT

Back-door attack poses a severe threat to deep learning systems. It injects hidden malicious behaviors to a model such that any input stamped with a special pattern can trigger such behaviors. Detecting back-door is hence of pressing need. Many existing defense techniques use optimization to generate the smallest input pattern that forces the model to misclassify a set of benign inputs injected with the pattern to a target label. However, the complexity is quadratic to the number of class labels such that they can hardly handle models with many classes. Inspired by Multi-Arm Bandit in Reinforcement Learning, we propose a K-Arm optimization method for backdoor detection. By iteratively and stochastically selecting the most promising labels for optimization with the guidance of an objective function, we substantially reduce the complexity, allowing to handle models with many classes. Moreover, by iteratively refining the selection of labels to optimize, it substantially mitigates the uncertainty in choosing the right labels, improving detection accuracy. At the time of submission, the evaluation of our method on over 4000 models in the IARPA TrojAI competition from round 1 to the latest round 4 achieves top performance on the leaderboard. Our technique also supersedes three state-of-the-art techniques in terms of accuracy and the scanning time needed.

Motivation & Objective

• Motivate robust detection of backdoors in deep neural networks to protect DL systems.
• Reduce the computational complexity of scanning across many classes and label-pairs.
• Develop a stochastic, round-based optimization framework to identify true backdoors while distinguishing natural features from triggers.
• Demonstrate scalability and effectiveness on large-scale TrojAI datasets and ImageNet models.

Proposed method

• Formulate backdoor scanning as a K-Arm optimization problem with K arms corresponding to target labels (and label-pairs for universal vs. label-specific backdoors).
• Use a trigger optimizer that stamps a pattern via a pattern P and mask M and optimizes min_P,M L(t,F((1−M)·X+M·P)) + α||M||1 with success condition Acc(Ẋ,t) ≥ θ and ||M||1 < previous ||M||1.
• Employ an ε-greedy K-Arm scheduler to balance exploration and exploitation, with an objective A(l) based on trigger size reduction rate and current trigger size.
• Incorporate pre-screening to reduce the arm set by filtering promising labels using logits statistics across benign samples.
• Extend the scheduler with symmetric optimization to distinguish natural features from true backdoors by comparing optimization directions between victim→target and target→victim.
• Provide theoretical analysis showing improved efficiency and provide empirical validation on TrojAI rounds 1–4 and ImageNet.

Experimental results

Research questions

• RQ1 Can a K-Arm bandit approach efficiently identify backdoors across models with many classes without exhaustively scanning every label pair?
• RQ2 Does stochastic, round-based label selection improve detection accuracy and reduce scanning time compared to existing methods like NC and ABS?
• RQ3 How effectively can symmetry considerations separate natural features from real backdoor triggers in practical datasets?
• RQ4 What are the empirical gains in accuracy and time when validating on large TrojAI rounds and ImageNet models?

Key findings

• Achieves top performance on the TrojAI leaderboard across rounds 1–4, outperforming NC, ABS, and ULP.
• Demonstrates significantly reduced scanning time, outperforming optimization-based baselines by substantial margins (e.g., 31% vs NC, 20% vs ABS, 27% vs ULP in accuracy).
• Handles models with many classes by reducing the effective arm space via pre-screening without sacrificing detection accuracy.
• Effectively differentiates between natural features and backdoor triggers through symmetric optimization and directional analysis.
• Shows strong results on ImageNet and other datasets with substantial reductions in runtime while maintaining high detection efficacy.