[Paper Review] Synthesizing Robust Adversarial Examples

Anish Athalye, Logan Engstrom+2 more|arXiv (Cornell University)|Jul 24, 2017

Adversarial Robustness in Machine Learning33 references684 citations

TL;DR

The paper presents Expectation Over Transformation (EOT), a general framework to synthesize adversarial examples robust to a distribution of transformations, and demonstrates 2D and 3D adversarial objects that fool classifiers in the physical world.

ABSTRACT

Standard methods for generating adversarial examples for neural networks do not consistently fool neural network classifiers in the physical world due to a combination of viewpoint shifts, camera noise, and other natural transformations, limiting their relevance to real-world systems. We demonstrate the existence of robust 3D adversarial objects, and we present the first algorithm for synthesizing examples that are adversarial over a chosen distribution of transformations. We synthesize two-dimensional adversarial images that are robust to noise, distortion, and affine transformation. We apply our algorithm to complex three-dimensional objects, using 3D-printing to manufacture the first physical adversarial objects. Our results demonstrate the existence of 3D adversarial objects in the physical world.

Motivation & Objective

• Motivate the need for adversarial examples that survive real-world transformations (viewpoint, lighting, noise).
• Introduce a general framework (EOT) to optimize adversarial inputs over a distribution of transformations.
• Demonstrate synthesis of robust 2D adversarial images and 3D textured objects, including physical fabrication via 3D printing.
• Show that robust adversarial objects can fool classifiers across varied viewpoints in the physical world.

Proposed method

• Introduce Expectation Over Transformation (EOT): maximize the expected log-probability of a target class over a distribution of transformations T.
• Define the objective as argmax_x' E_{t~T}[log P(y_t|t(x'))] subject to E_{t~T}[d(t(x'),t(x))] < ε, where d is a perceptual distance.
• Approximate the gradient of the expectation by stochastic sampling of transformations during each SGD step and differentiate through the transformation.
• Use a Lagrangian-relaxed form to combine the objective with a perceptual distance term in LAB color space to encourage imperceptibility.
• In 2D, model T with affine-like transformations (rotation, translation, noise, lighting); in 3D, treat textures as inputs and render their views under a pose distribution, differentiating through the rendering process.
• Fabricate physical 3D adversarial objects by applying EOT to textures on 3D models and printing them, accounting for real-world printing and lighting variations.

Experimental results

Research questions

• RQ1 Can adversarial examples be made robust to a distribution of real-world transformations?
• RQ2 Can the EOT framework synthesize 2D and 3D adversarial inputs that remain adversarial under diverse viewpoints and lighting?
• RQ3 Is it possible to fabricate physical 3D objects that fool classifiers across a range of poses and conditions?
• RQ4 How do perceptual (LAB) distance and transformation distributions affect perturbation size and robustness?

Key findings

• 2D adversarial examples can achieve high robustness across 1000 random transformations (mean adversariality around 96.4%).
• 3D textured objects can be made adversarial across 100 random poses with mean adversariality around 83.4%.
• Two printed 3D objects (turtle and baseball) remain adversarial over a wide distribution of viewpoints in physical tests (e.g., turtle adversariality 82% over 100 photos).
• The method demonstrates that defenses based on input transformations do not reliably stop robust adversarial examples (EOT can bypass several defenses).
• Printing color inaccuracies and lighting variations were modeled in the transformation distribution, yet robust adversarial objects were still produced.