[Paper Review] BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detection

Tinghao Xie, Xiangyu Qi+4 more|arXiv (Cornell University)|Aug 23, 2023

Adversarial Robustness in Machine Learning4 citations

TL;DR

BaDExpert proposes a post-development defense that extracts backdoor functionality into a dedicated backdoor expert model via fine-tuning on a small set of mislabeled clean samples. This expert model detects backdoor inputs by disagreement with the original model, achieving high AUROC (up to 99.8%) and mitigating 17 SOTA backdoor attacks with minimal clean accuracy drop across CIFAR10, GTSRB, and ImageNet.

ABSTRACT

We present a novel defense, against backdoor attacks on Deep Neural Networks (DNNs), wherein adversaries covertly implant malicious behaviors (backdoors) into DNNs. Our defense falls within the category of post-development defenses that operate independently of how the model was generated. The proposed defense is built upon a novel reverse engineering approach that can directly extract backdoor functionality of a given backdoored model to a backdoor expert model. The approach is straightforward -- finetuning the backdoored model over a small set of intentionally mislabeled clean samples, such that it unlearns the normal functionality while still preserving the backdoor functionality, and thus resulting in a model (dubbed a backdoor expert model) that can only recognize backdoor inputs. Based on the extracted backdoor expert model, we show the feasibility of devising highly accurate backdoor input detectors that filter out the backdoor inputs during model inference. Further augmented by an ensemble strategy with a finetuned auxiliary model, our defense, BaDExpert (Backdoor Input Detection with Backdoor Expert), effectively mitigates 17 SOTA backdoor attacks while minimally impacting clean utility. The effectiveness of BaDExpert has been verified on multiple datasets (CIFAR10, GTSRB and ImageNet) across various model architectures (ResNet, VGG, MobileNetV2 and Vision Transformer).

Motivation & Objective

• To address the challenge of detecting backdoor inputs in post-development deep neural networks without access to training data or training dynamics.
• To develop a defense that is agnostic to how the model was trained, including both data-poisoning and weight-tampering attacks.
• To extract the backdoor functionality directly, avoiding reliance on trigger pattern reconstruction that fails under global transformations or complex triggers.
• To achieve robust backdoor input detection with high accuracy and minimal impact on clean model utility.
• To demonstrate effectiveness across diverse datasets, model architectures, and 17 state-of-the-art backdoor attacks.

Proposed method

• Fine-tune the backdoored model on a small, intentionally mislabeled clean dataset to unlearn normal classification while preserving backdoor functionality.
• Train a backdoor expert model that only recognizes backdoor inputs by leveraging the model's tendency to lose clean accuracy but retain attack success rate during this fine-tuning.
• Use the backdoor expert model to detect backdoor inputs at inference time by checking for prediction disagreement with the original model.
• Implement an ensemble strategy combining the backdoor expert with a finetuned auxiliary model to improve detection robustness and AUROC.
• Apply the detection pipeline to multiple datasets (CIFAR10, GTSRB, ImageNet) and model architectures (ResNet, VGG, MobileNetV2, Vision Transformer).
• Use AUROC as the primary metric to evaluate detection performance across various attack types.

Experimental results

Research questions

• RQ1 Can backdoor functionality be reliably extracted from a backdoored model without access to training data or gradients?
• RQ2 Does fine-tuning on mislabeled clean samples effectively isolate backdoor functionality while suppressing normal classification?
• RQ3 Can a backdoor expert model built via this method serve as an effective detector for backdoor inputs at inference time?
• RQ4 How does the ensemble of backdoor experts and auxiliary models improve detection robustness across diverse attack types?
• RQ5 To what extent does the defense preserve clean accuracy while mitigating state-of-the-art backdoor attacks?

Key findings

• BaDExpert achieves an AUROC of 99.8% on CIFAR10 under BadNet attacks, significantly outperforming existing detectors.
• The method mitigates 17 state-of-the-art backdoor attacks across CIFAR10, GTSRB, and ImageNet with minimal clean accuracy drop (≤0.5%).
• The backdoor expert model maintains high attack success rate (≥95%) while clean accuracy drops below 1% after fine-tuning.
• The ensemble strategy improves detection robustness, with standard deviations of AUROC across experiments consistently below 1.0% for most attacks.
• The approach is effective across diverse model architectures, including Vision Transformers, and remains robust under global transformation-based triggers where prior methods fail.
• Standard deviation analysis confirms high reproducibility, with AUROC standard deviations below 1.0% for most attacks in major experiments.