[Paper Review] NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles

Jiajun Lu, Hussein Sibai+2 more|arXiv (Cornell University)|Jul 12, 2017

Adversarial Robustness in Machine Learning17 references209 citations

TL;DR

The paper argues that physical adversarial perturbations fail to consistently fool object detectors in autonomous vehicles across varying viewing distances and angles.

ABSTRACT

It has been shown that most machine learning algorithms are susceptible to adversarial perturbations. Slightly perturbing an image in a carefully chosen direction in the image space may cause a trained neural network model to misclassify it. Recently, it was shown that physical adversarial examples exist: printing perturbed images then taking pictures of them would still result in misclassification. This raises security and safety concerns. However, these experiments ignore a crucial property of physical objects: the camera can view objects from different distances and at different angles. In this paper, we show experiments that suggest that current constructions of physical adversarial examples do not disrupt object detection from a moving platform. Instead, a trained neural network classifies most of the pictures taken from different distances and angles of a perturbed image correctly. We believe this is because the adversarial property of the perturbation is sensitive to the scale at which the perturbed picture is viewed, so (for example) an autonomous car will misclassify a stop sign only from a small range of distances. Our work raises an important question: can one construct examples that are adversarial for many or most viewing conditions? If so, the construction should offer very significant insights into the internal representation of patterns by deep networks. If not, there is a good prospect that adversarial examples can be reduced to a curiosity with little practical impact.

Motivation & Objective

• Motivate and quantify whether physical adversarial examples threaten object detectors in autonomous driving.
• Extend adversarial attacks from classifiers to detectors like YOLO.
• Assess how distance and viewing angle impact the effectiveness of physical adversarial perturbations.

Proposed method

• Apply fast sign, iterative, and L-BFGS adversarial attacks to both a traffic sign classifier and a YOLO detector.
• Generate adversarial stop-sign images and print them; simulate driving by capturing photos at multiple distances and angles.
• Use destruction rate to measure how often adversarial examples remain adversarial after transformations (printing and recapturing).
• Conduct controlled experiments with printed signs at 0.5 m and 1.5 m and real-world driving scenarios to evaluate detector and classifier robustness.

Experimental results

Research questions

• RQ1 Do physical adversarial perturbations remain effective against object detectors (e.g., YOLO) when the viewing distance and angle change?
• RQ2 How does distance influence the destruction rate of adversarial perturbations in both detector and classifier tasks?
• RQ3 Can adversarial perturbations crafted for detectors transfer to or persist against classifiers under physical-world conditions?
• RQ4 What practical implications do distance and angle effects have for safety in autonomous driving?

Key findings

• Physical adversarial perturbations largely fail to fool the YOLO detector across different distances; destruction rates rise with distance.
• For the detector, many perturbed signs are correctly detected at 0.5 m and especially at 1.5 m, despite adversarial generation.
• Classifier attacks show higher destruction at greater distances, with varying effectiveness by method (Iterative and LBFGS generally more damaging than Fast Sign but still degrade with distance).
• Cropping and background context can influence detection outcomes, sometimes improving detection with higher contrast backgrounds in printed tests.
• Angle of capture can alter adversarial effectiveness, suggesting real-world viewing conditions reduce reliable adversarial impact.