[Paper Review] Untargeted Backdoor Attack against Object Detection

Chengxiao Luo, Yiming Li+2 more|arXiv (Cornell University)|Nov 2, 2022

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper proposes a stealthy, poison-only untargeted backdoor attack against object detection models by removing bounding boxes of objects after injecting trigger patterns, achieving high effectiveness in both digital and physical settings while remaining resistant to common defenses. The attack causes models to fail detecting objects with triggers, while maintaining normal performance on benign samples.

ABSTRACT

Recent studies revealed that deep neural networks (DNNs) are exposed to backdoor threats when training with third-party resources (such as training samples or backbones). The backdoored model has promising performance in predicting benign samples, whereas its predictions can be maliciously manipulated by adversaries based on activating its backdoors with pre-defined trigger patterns. Currently, most of the existing backdoor attacks were conducted on the image classification under the targeted manner. In this paper, we reveal that these threats could also happen in object detection, posing threatening risks to many mission-critical applications ($e.g.$, pedestrian detection and intelligent surveillance systems). Specifically, we design a simple yet effective poison-only backdoor attack in an untargeted manner, based on task characteristics. We show that, once the backdoor is embedded into the target model by our attack, it can trick the model to lose detection of any object stamped with our trigger patterns. We conduct extensive experiments on the benchmark dataset, showing its effectiveness in both digital and physical-world settings and its resistance to potential defenses.

Motivation & Objective

• To reveal the vulnerability of object detection models to backdoor attacks when trained with third-party or untrusted data.
• To design a poison-only, untargeted backdoor attack that evades detection of objects marked with specific trigger patterns.
• To ensure the attack remains stealthy by preserving model accuracy on benign samples while degrading performance only on poisoned samples.
• To evaluate the attack's effectiveness in both digital and physical-world scenarios, including real-world deployment with printed triggers.
• To test resistance against common backdoor defenses such as fine-tuning and model pruning.

Proposed method

• The attack injects a predefined trigger pattern at the center of randomly selected objects in training images.
• After trigger injection, the bounding box annotations for those objects are removed by setting their width and height to zero.
• This creates a latent backdoor where the model learns to ignore objects containing the trigger, without altering model architecture or loss function.
• The poisoned training set is used to retrain the victim detector, preserving performance on benign data.
• During inference, the trigger activates the backdoor, causing the model to fail to detect the object.
• The method is evaluated under both digital (direct image modification) and physical (printed trigger patches) settings.

Fig. 1 : The main pipeline of our poison-only untargeted backdoor attack against object detection. Our method consists of three main stages. In the first stage, we generate some poisoned training samples by adding trigger patterns to randomly selected benign samples and reassigning the width and hei

Experimental results

Research questions

• RQ1 Can backdoor attacks be effectively mounted on object detection models using only data poisoning, without access to model architecture or training procedure?
• RQ2 How effective is the attack in both digital and physical-world deployment scenarios?
• RQ3 To what extent does the attack remain stealthy, maintaining high accuracy on benign samples while degrading performance on poisoned samples?
• RQ4 How resistant is the attack to standard backdoor defense techniques such as fine-tuning and model pruning?
• RQ5 Does the choice of trigger pattern significantly affect the attack's success rate?

Key findings

• The attack reduces mAP on poisoned datasets to below 11% across all evaluated models, while maintaining mAP on benign datasets within 1-2% of the original model.
• The attack achieves over 30% drop in AP50 on poisoned datasets, even when the model's performance on the poisoned test set remains stable.
• The attack remains effective in physical-world settings, with models failing to detect objects stamped with printed trigger patches captured via smartphone camera.
• Different trigger patterns (e.g., square, cross, circle, star) yield similar attack success rates, indicating robustness to trigger design.
• The attack is resistant to fine-tuning: even after fine-tuning on 10% benign samples, poisoned mAP remains below 15%.
• Model pruning further reduces the poisoned mAP, indicating the attack is not only resistant but potentially amplified by pruning defenses.

Fig. 2 : The detection results of the Sparse R-CNN model under our attack in the digital space. In these examples, the trigger patterns are added by pixel-wise replacement of objects ( $i.e.$ , ‘person’ and ‘stop sign’) in the digital space.