[Paper Review] Few-shot Backdoor Defense Using Shapley Estimation

Jiyang Guan, Zhuozhuo Tu+2 more|arXiv (Cornell University)|Dec 30, 2021

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper proposes Shapley Pruning (ShapPruning), a few-shot backdoor defense method that uses Shapley value estimation to identify and prune only the most influential neurons responsible for backdoor behavior. By leveraging model activation patterns and a novel acceleration strategy, it achieves robust defense with minimal clean data (as few as one image per class) and prunes less than 1% of neurons, maintaining model accuracy within 1% decline while reducing attack success rates to below 0.5%.

ABSTRACT

Deep neural networks have achieved impressive performance in a variety of tasks over the last decade, such as autonomous driving, face recognition, and medical diagnosis. However, prior works show that deep neural networks are easily manipulated into specific, attacker-decided behaviors in the inference stage by backdoor attacks which inject malicious small hidden triggers into model training, raising serious security threats. To determine the triggered neurons and protect against backdoor attacks, we exploit Shapley value and develop a new approach called Shapley Pruning (ShapPruning) that successfully mitigates backdoor attacks from models in a data-insufficient situation (1 image per class or even free of data). Considering the interaction between neurons, ShapPruning identifies the few infected neurons (under 1% of all neurons) and manages to protect the model's structure and accuracy after pruning as many infected neurons as possible. To accelerate ShapPruning, we further propose discarding threshold and $ε$-greedy strategy to accelerate Shapley estimation, making it possible to repair poisoned models with only several minutes. Experiments demonstrate the effectiveness and robustness of our method against various attacks and tasks compared to existing methods.

Motivation & Objective

• Address the challenge of defending against backdoor attacks in data-insufficient scenarios, where only a few clean samples or none are available.
• Accurately identify and isolate the minimal set of infected neurons responsible for backdoor behavior, rather than relying on heuristic or global pruning.
• Maintain high model accuracy and structural integrity after mitigation, avoiding excessive pruning that degrades performance.
• Accelerate Shapley value estimation for real-world applicability, enabling defense in minutes rather than hours.
• Enable data-free backdoor mitigation by reconstructing triggers and clean data from model statistics, particularly batch normalization layers.

Proposed method

• Use Shapley values from cooperative game theory to attribute the overall backdoor behavior to individual neurons, identifying those most responsible for malicious predictions.
• Introduce a trigger reverse method to reconstruct poisoned inputs from model activations, enabling accurate Shapley estimation even without clean data.
• Propose a discarding threshold and ε-greedy strategy to accelerate Shapley estimation by focusing on high-impact neurons and balancing exploration and exploitation.
• Implement a mixture-mode ShapPruning framework that combines data-free and few-shot settings, using batch normalization statistics to improve data reconstruction quality.
• Apply neuron pruning based on Shapley values to remove only the most infected neurons, preserving model structure and accuracy.
• Use Monte Carlo estimation with early stopping and confidence bounds to reduce computation time while maintaining estimation accuracy.

Experimental results

Research questions

• RQ1 Can Shapley value estimation be effectively applied to detect and isolate backdoor-triggering neurons in deep neural networks with minimal clean data?
• RQ2 How can Shapley value estimation be accelerated to make backdoor mitigation practical for real-world models?
• RQ3 Can backdoor mitigation be achieved in a data-free setting by leveraging internal model statistics such as batch normalization layers?
• RQ4 To what extent does ShapPruning preserve model accuracy while removing backdoor behavior compared to existing defense methods?
• RQ5 How robust is ShapPruning across different attack types (e.g., BadNets, Trojan, Physical Key) and model architectures (e.g., VGG, ResNet)?

Key findings

• ShapPruning successfully mitigates backdoor attacks using only one image per class, achieving a 0.1% accuracy decline on the GTSRB dataset.
• The attack success rate drops to 0.4% after pruning, demonstrating strong defense efficacy with minimal performance cost.
• The method prunes less than 1% of all neurons on average, significantly reducing the number of neurons removed compared to prior methods like Fine Pruning (70% pruning).
• The proposed acceleration strategy (discarding threshold and ε-greedy) identifies the top-50 neurons with the highest Shapley values with 46/50 correct matches in the top-70 of the ground truth, outperforming T-MAB (27/50 matches).
• In data-free settings, ShapPruning achieves better backdoor mitigation than DeepInspect and other baselines, thanks to improved trigger reconstruction using batch normalization statistics.
• The method is robust across multiple datasets (CIFAR10, CIFAR100), architectures (VGG16, ResNet34), and attack types (BadNets, Trojan, Physical Key), consistently reducing attack success rates to near-zero levels.