[Paper Review] Robust Physical-World Attacks on Deep Learning Models

Kevin Eykholt, Ivan Evtimov+7 more|arXiv (Cornell University)|Jul 27, 2017

Adversarial Robustness in Machine Learning39 references506 citations

TL;DR

The paper introduces Robust Physical Perturbations (RP2) to create perturbations on physical objects that cause targeted misclassification of DNNs under varying viewing distances and angles, and evaluates them in lab and field tests on road signs and other objects.

ABSTRACT

Recent studies show that the state-of-the-art deep neural networks (DNNs) are vulnerable to adversarial examples, resulting from small-magnitude perturbations added to the input. Given that that emerging physical systems are using DNNs in safety-critical situations, adversarial examples could mislead these systems and cause dangerous situations.Therefore, understanding adversarial examples in the physical world is an important step towards developing resilient learning algorithms. We propose a general attack algorithm,Robust Physical Perturbations (RP2), to generate robust visual adversarial perturbations under different physical conditions. Using the real-world case of road sign classification, we show that adversarial examples generated using RP2 achieve high targeted misclassification rates against standard-architecture road sign classifiers in the physical world under various environmental conditions, including viewpoints. Due to the current lack of a standardized testing method, we propose a two-stage evaluation methodology for robust physical adversarial examples consisting of lab and field tests. Using this methodology, we evaluate the efficacy of physical adversarial manipulations on real objects. Witha perturbation in the form of only black and white stickers,we attack a real stop sign, causing targeted misclassification in 100% of the images obtained in lab settings, and in 84.8%of the captured video frames obtained on a moving vehicle(field test) for the target classifier.

Motivation & Objective

• Demonstrate that physical perturbations can reliably mislead DNN classifiers in real-world, dynamic conditions.
• Develop RP2 to generate perturbations robust to varying distances, angles, and lighting.
• Propose a two-stage lab-and-field evaluation methodology for physical adversarial examples.
• Evaluate perturbations on standard road-sign classifiers and show generalization to other objects.

Proposed method

• Model a distribution of physical transformations (distance, angle, lighting) and sample real and synthetic variations to optimize perturbations.
• Use a mask Mx to constrain perturbations to the surface of the target object and project perturbations via T_i to align with object transformations.
• Incorporate a Non-Printability Score (NPS) to account for printer color reproduction errors in the optimization objective.
• Optimize perturbations by solving a relaxed objective with Lp regularization and expectation over transformed instances: argmin_delta lambda||Mx·delta||p + NPS + E_{xi~XV} J(f_theta(xi + Ti(Mx·delta)), y*) .
• Employ ADAM optimization and craft perturbations as black-and-white stickers or graffiti-like posters on stop signs.

Experimental results

Research questions

• RQ1 Can physically applied perturbations on real objects cause targeted misclassification across a range of distances and viewing angles?
• RQ2 Do robust, surface-constrained perturbations maintain effectiveness under environmental variability and fabrication limitations?
• RQ3 How do lab (stationary) and field (drive-by) tests compare in assessing physical adversarial perturbations?
• RQ4 Are the RP2 perturbations transferable to other classifiers and objects beyond road signs?
• RQ5 What is the impact of perturbation type (poster vs. sticker) on attack success and visibility?

Key findings

• RP2 perturbations achieve 100% targeted success on stationary Stop signs for LISA-CNN using poster attacks.
• Drive-by tests show 84.8% targeted success with camouflage graffiti on LISA-CNN and 87.5% on GTSRB-CNN for drive-by frames.
• Poster and sticker attacks yield high targeted success across distances up to 40 feet and angles up to 60 degrees in lab tests.
• On Inception-v3, a sticker attack misclassifies a microwave as a phone with 90% targeted success, and a coffee mug as a cash machine with 71.4% targeted success.
• GTSRB-CNN shows 80% targeted success for Stop vs Speed Limit 80 in stationary tests and 87.5% in drive-by tests.
• The approach generalizes to other objects beyond road signs, demonstrating broad susceptibility of image classifiers to robust physical perturbations.