[Paper Review] Constructing Unrestricted Adversarial Examples with Generative Models

Yang Song, Rui Shu+2 more|arXiv (Cornell University)|May 21, 2018

Adversarial Robustness in Machine LearningComputer Science49 references125 citations

TL;DR

The paper introduces unrestricted adversarial examples synthesized from scratch using conditional generative models (AC-GAN), and shows they can bypass certified defenses and adversarial training while remaining legitimate to humans.

ABSTRACT

Adversarial examples are typically constructed by perturbing an existing data point within a small matrix norm, and current defense methods are focused on guarding against this type of attack. In this paper, we propose unrestricted adversarial examples, a new threat model where the attackers are not restricted to small norm-bounded perturbations. Different from perturbation-based attacks, we propose to synthesize unrestricted adversarial examples entirely from scratch using conditional generative models. Specifically, we first train an Auxiliary Classifier Generative Adversarial Network (AC-GAN) to model the class-conditional distribution over data samples. Then, conditioned on a desired class, we search over the AC-GAN latent space to find images that are likely under the generative model and are misclassified by a target classifier. We demonstrate through human evaluation that unrestricted adversarial examples generated this way are legitimate and belong to the desired class. Our empirical results on the MNIST, SVHN, and CelebA datasets show that unrestricted adversarial examples can bypass strong adversarial training and certified defense methods designed for traditional adversarial attacks.

Motivation & Objective

• Motivate a threat model where adversarial inputs are not limited to small perturbations of existing data.
• Propose a practical method to generate unrestricted adversarial examples from scratch using class-conditional generative models.
• Evaluate the attack's effectiveness against certified defenses and adversarially trained models across multiple datasets.

Proposed method

• Train an Auxiliary Classifier GAN (AC-GAN) to model class-conditional image distributions.
• Search the AC-GAN latent space to find images that are likely under the model and misclassified by a target classifier.
• Optimize a loss L = L0 + λ1 L1 + λ2 L2 to produce high-fidelity unrestricted adversarial examples, where L0 targets the classifier, L1 regularizes the latent code, and L2 aligns with the source class via the auxiliary classifier.
• Optionally augment generated images with small trainable noise to enhance diversity (noise-augmented attack).
• Use Amazon Mechanical Turk to validate that synthesized images belong to the intended class and are legitimate to humans.

Experimental results

Research questions

• RQ1 Can unrestricted adversarial examples—images synthesized from scratch—mislead classifiers despite strong defenses?
• RQ2 How effective are AC-GAN-based unrestricted adversarial examples against certified defenses and adversarial training across MNIST, SVHN, and CelebA?
• RQ3 Do unrestricted adversarial examples transfer to other architectures in black-box settings?
• RQ4 Does adding noise to the generator improve attack effectiveness or realism?

Key findings

• Unrestricted adversarial examples achieve high success rates (over 84%) across MNIST, SVHN, and CelebA datasets in fooling target classifiers.
• The attacks bypass certified defenses designed for perturbation-based attacks and also threaten adversarially trained models.
• Human evaluation via MTurk confirms that many unrestricted adversarial examples are legitimate images belonging to the target class.
• Noise-augmented variants can improve transferability to some models while affecting others variably.
• Unrestricted adversarial examples show moderate transferability to other architectures, indicating potential black-box risk.