[Paper Review] NTD: Non-Transferability Enabled Backdoor Detection

Yinshan Li, Hua Ma+7 more|arXiv (Cornell University)|Nov 22, 2021

Adversarial Robustness in Machine Learning38 references4 citations

TL;DR

NTD proposes a lightweight, user-friendly online backdoor detection method that leverages the intrinsic non-transferability of backdoor triggers to detect malicious inputs at inference time. By comparing input features with those of benign samples from the same predicted class using a private, pre-trained feature extractor, NTD achieves high detection accuracy with minimal computational overhead and no need for ML expertise or training data access.

ABSTRACT

A backdoor deep learning (DL) model behaves normally upon clean inputs but misbehaves upon trigger inputs as the backdoor attacker desires, posing severe consequences to DL model deployments. State-of-the-art defenses are either limited to specific backdoor attacks (source-agnostic attacks) or non-user-friendly in that machine learning (ML) expertise or expensive computing resources are required. This work observes that all existing backdoor attacks have an inevitable intrinsic weakness, non-transferability, that is, a trigger input hijacks a backdoored model but cannot be effective to another model that has not been implanted with the same backdoor. With this key observation, we propose non-transferability enabled backdoor detection (NTD) to identify trigger inputs for a model-under-test (MUT) during run-time.Specifically, NTD allows a potentially backdoored MUT to predict a class for an input. In the meantime, NTD leverages a feature extractor (FE) to extract feature vectors for the input and a group of samples randomly picked from its predicted class, and then compares similarity between the input and the samples in the FE's latent space. If the similarity is low, the input is an adversarial trigger input; otherwise, benign. The FE is a free pre-trained model privately reserved from open platforms. As the FE and MUT are from different sources, the attacker is very unlikely to insert the same backdoor into both of them. Because of non-transferability, a trigger effect that does work on the MUT cannot be transferred to the FE, making NTD effective against different types of backdoor attacks. We evaluate NTD on three popular customized tasks such as face recognition, traffic sign recognition and general animal classification, results of which affirm that NDT has high effectiveness (low false acceptance rate) and usability (low false rejection rate) with low detection latency.

Motivation & Objective

• To address the limitations of existing backdoor defenses that are either trigger-type specific or require high ML expertise and computational resources.
• To develop a practical, user-friendly defense that operates at inference time without access to training data or model weights.
• To exploit the intrinsic property of non-transferability in backdoor attacks to detect trigger inputs effectively across diverse backdoor types.
• To enable real-time detection with low latency and high accuracy in security-sensitive applications like facial recognition and autonomous driving.

Proposed method

• NTD uses a privately reserved, pre-trained feature extractor (FE) from a reputable source to extract latent features from input samples.
• For a given input, NTD computes its feature similarity to a set of randomly sampled benign samples from the same predicted class in the FE's latent space.
• If the similarity between the input and the benign samples is low, the input is classified as a trigger input; otherwise, it is deemed benign.
• The method relies on the non-transferability property: a trigger effective on the model-under-test (MUT) will not work on the FE, which is independently trained and not backdoored.
• To reduce online latency, NTD pre-computes and caches all feature vectors from the validation set during an offline phase.
• The detection mechanism supports both global and per-class thresholds, with the latter enhancing protection for high-risk target classes.

Experimental results

Research questions

• RQ1 Can a backdoor detection method effectively detect trigger inputs without requiring access to training data or model weights?
• RQ2 Can non-transferability of backdoor triggers be exploited to build a detection mechanism that is agnostic to both trigger type and backdoor type?
• RQ3 Is it possible to design a lightweight, online detection system that requires no ML expertise and minimal computational overhead?
• RQ4 How does the detection performance of NTD compare to existing defenses in terms of false acceptance rate (FAR) and false rejection rate (FRR)?

Key findings

• NTD achieves a low false acceptance rate (FAR) across multiple tasks, indicating high effectiveness in detecting backdoored inputs.
• The method maintains a low false rejection rate (FRR), demonstrating strong robustness against clean inputs.
• NTD operates with low online detection latency, as feature vectors are pre-computed and cached from the validation set.
• The detection mechanism is agnostic to both trigger type and backdoor type, including source-agnostic and source-specific attacks.
• NTD outperforms existing defenses in usability, requiring no ML expertise, no training, and no access to training data.
• The use of a reputable, independent feature extractor ensures that the FE remains immune to backdoors in the MUT, enabling reliable detection.