[Paper Review] Daedalus: Breaking Non-Maximum Suppression in Object Detection via Adversarial Examples

Derui Wang, Chaoran Li+5 more|arXiv (Cornell University)|Feb 6, 2019

Adversarial Robustness in Machine Learning71 references35 citations

TL;DR

Daedalus crafts adversarial examples that disable NMS in end-to-end object detectors, massively increasing false positives and driving mAP to zero under certain settings; it can be realized in physical posters and works across multiple models.

ABSTRACT

This paper demonstrates that Non-Maximum Suppression (NMS), which is commonly used in Object Detection (OD) tasks to filter redundant detection results, is no longer secure. Considering that NMS has been an integral part of OD systems, thwarting the functionality of NMS can result in unexpected or even lethal consequences for such systems. In this paper, an adversarial example attack which triggers malfunctioning of NMS in end-to-end OD models is proposed. The attack, namely exttt{Daedalus}, compresses the dimensions of detection boxes to evade NMS. As a result, the final detection output contains extremely dense false positives. This can be fatal for many OD applications such as autonomous vehicles and surveillance systems. The attack can be generalised to different end-to-end OD models, such that the attack cripples various OD applications. Furthermore, a way to craft robust adversarial examples is developed by using an ensemble of popular detection models as the substitutes. Considering the pervasive nature of model reusing in real-world OD scenarios, Daedalus examples crafted based on an extit{ensemble of substitutes} can launch attacks without knowing the parameters of the victim models. Experimental results demonstrate that the attack effectively stops NMS from filtering redundant bounding boxes. As the evaluation results suggest, Daedalus increases the false positive rate in detection results to $99.9\%$ and reduces the mean average precision scores to $0$, while maintaining a low cost of distortion on the original inputs. It is also demonstrated that the attack can be practically launched against real-world OD systems via printed posters.

Motivation & Objective

• Motivate and demonstrate that Non-Maximum Suppression (NMS) can be broken by adversarial inputs in end-to-end object detectors.
• Show that breaking NMS degrades detection performance and can be realized as a universal, transferable attack.
• Develop an ensemble-substitute strategy to craft robust adversarial examples that work across multiple models.
• Explore practical, physical-world deployment of the attack using printed posters.

Proposed method

• Formulate Daedalus as a constrained optimization to maximize adversarial objectives while bounding perturbations.
• Exploit NMS vulnerability by reducing IoU among bounding boxes or compressing box dimensions to prevent NMS filtering.
• Propose three adversarial loss functions (f1, f2, f3) to induce dense, overlapping, or small boxes that evade NMS.
• Use an ensemble of substitute detectors (e.g., YOLO-v3, SSD, RetinaNet) to generate universal adversarial examples for black-box scenarios.
• Provide L2 and L0 variants of Daedalus to study all-pixel and selective-pixel perturbations, including a physical poster realization.

Experimental results

Research questions

• RQ1 Can NMS be reliably broken by carefully crafted adversarial perturbations in end-to-end object detectors?
• RQ2 Do universal adversarial examples generated from an ensemble of substitute models transfer to other detectors in black-box settings?
• RQ3 How do Daedalus perturbations affect standard OD metrics like FP rate and mAP across varying NMS thresholds and attack confidences?
• RQ4 Is it feasible to deploy Daedalus in real-world scenarios via printed posters or other physical media?

Key findings

• Daedalus can cause extremely dense false positives by preventing NMS from filtering outputs.
• Under evaluation, FP rate reaches up to 99.9% for high-confidence attacks, while mAP can drop to 0% under certain settings.
• An ensemble-substitute approach yields universal adversarial examples that can attack multiple popular OD models.
• The attack is demonstrated in a physical poster setting to attack real-world OD systems.