[Paper Review] Evading Real-Time Person Detectors by Adversarial T-shirt.

Kaidi Xu, Gaoyuan Zhang+7 more|arXiv (Cornell University)|Oct 18, 2019

Adversarial Robustness in Machine Learning19 references18 citations

TL;DR

This paper proposes an adversarial T-shirt that evades real-time person detectors despite deformation from body movements, using deformation-aware optimization to maintain attack efficacy. It achieves 79% success in simulation and 63% in physical deployment against YOLOv2, outperforming prior state-of-the-art methods by over 35 percentage points.

ABSTRACT

It is known that deep neural networks (DNNs) could be vulnerable to adversarial attacks. The so-called physical adversarial examples deceive DNN-based decision makers by attaching adversarial patches to real objects. However, most of the existing works on physical adversarial attacks focus on static objects such as glass frame, stop sign and image attached to a cardboard. In this work, we proposed adversarial T-shirt, a robust physical adversarial example for evading person detectors even if it suffers from deformation due toa moving person's pose change. To the best of our knowledge, the effect of deformation is first modeled for designing physical adversarial examples with respect to non-rigid objects such as T-shirts. We show that the proposed method achieves 79% and 63% attack success rates in digital and physical worlds respectively against YOLOv2. In contrast, the state-of-the-art physical attack method to fool a person detector only achieves 27% attack success rate. Furthermore, by leveraging min-max optimization, we extend our method to the ensemble attack setting against object detectors YOLOv2 and Faster R-CNN simultaneously.

Motivation & Objective

• To address the challenge of designing robust physical adversarial examples for non-rigid, deformable objects like T-shirts.
• To model the effect of pose-induced deformation in physical adversarial attacks on person detectors.
• To improve attack success rates in real-world deployment compared to existing methods targeting static objects.
• To extend the approach to ensemble attacks against multiple detectors, including YOLOv2 and Faster R-CNN.
• To demonstrate the feasibility of deploying effective adversarial examples on wearable clothing in dynamic environments.

Proposed method

• The method formulates a deformation-aware optimization process that accounts for pose variations in a moving person, modeling how the adversarial pattern on a T-shirt deforms across different body poses.
• It employs a min-max optimization framework to generate adversarial patterns that remain effective under various deformations, ensuring robustness during real-world deployment.
• The adversarial T-shirt is generated by optimizing the pattern on a 3D T-shirt mesh, simulating realistic deformations from human motion.
• The approach integrates a differentiable rendering pipeline to simulate how the adversarial pattern appears on a moving person from multiple camera viewpoints.
• It extends to ensemble attacks by jointly optimizing for multiple detectors (YOLOv2 and Faster R-CNN) using a min-max formulation.
• The method uses a differentiable differentiable renderer to backpropagate gradients through the image formation process, enabling end-to-end training of the adversarial pattern.

Experimental results

Research questions

• RQ1 Can adversarial patterns on wearable clothing like T-shirts maintain effectiveness under significant deformation due to human motion?
• RQ2 How does modeling deformation during adversarial pattern generation improve physical-world attack success rates compared to static object-based attacks?
• RQ3 Can the proposed method achieve high attack success on multiple object detectors simultaneously in the physical world?
• RQ4 What is the performance gap between digital simulation and real-world deployment for adversarial T-shirts?
• RQ5 How does the min-max optimization strategy improve robustness across diverse detectors and deformation scenarios?

Key findings

• The adversarial T-shirt achieves a 79% attack success rate in digital simulations against YOLOv2, significantly outperforming prior state-of-the-art methods.
• In physical-world deployment, the method attains a 63% attack success rate, which is over 35 percentage points higher than the previous best method.
• The proposed method demonstrates robustness to deformation caused by human motion, which is explicitly modeled during pattern generation.
• The min-max optimization framework enables effective ensemble attacks, successfully evading both YOLOv2 and Faster R-CNN simultaneously.
• The study provides the first evidence that deformation-aware design is critical for effective physical adversarial attacks on non-rigid, wearable objects.
• The results confirm that physical-world attack success is substantially higher when deformation is modeled during the attack generation process.