[Paper Review] Towards the first adversarially robust neural network model on MNIST

Lukas Schott, Jonas Rauber+2 more|arXiv (Cornell University)|May 23, 2018

Adversarial Robustness in Machine LearningComputer Science31 references86 citations

TL;DR

The paper argues MNIST is not yet adversarially robust and introduces Analysis by Synthesis (ABS) with class-conditional VAEs to achieve state-of-the-art robustness across L0, L2, and L∞ attacks, with semantically meaningful adversarial examples.

ABSTRACT

Despite much effort, deep neural networks remain highly susceptible to tiny input perturbations and even for MNIST, one of the most common toy datasets in computer vision, no neural network model exists for which adversarial perturbations are large and make semantic sense to humans. We show that even the widely recognized and by far most successful defense by Madry et al. (1) overfits on the L-infinity metric (it's highly susceptible to L2 and L0 perturbations), (2) classifies unrecognizable images with high certainty, (3) performs not much better than simple input binarization and (4) features adversarial perturbations that make little sense to humans. These results suggest that MNIST is far from being solved in terms of adversarial robustness. We present a novel robust classification model that performs analysis by synthesis using learned class-conditional data distributions. We derive bounds on the robustness and go to great length to empirically evaluate our model using maximally effective adversarial attacks by (a) applying decision-based, score-based, gradient-based and transfer-based attacks for several different Lp norms, (b) by designing a new attack that exploits the structure of our defended model and (c) by devising a novel decision-based attack that seeks to minimize the number of perturbed pixels (L0). The results suggest that our approach yields state-of-the-art robustness on MNIST against L0, L2 and L-infinity perturbations and we demonstrate that most adversarial examples are strongly perturbed towards the perceptual boundary between the original and the adversarial class.

Motivation & Objective

• Show that MNIST remains unsolved for adversarial robustness under current defenses.
• Propose a robust classification model based on analysis by synthesis (ABS) with class-conditional distributions.
• Provide instance-specific robustness guarantees and strong attack evaluation.
• Demonstrate that ABS yields semantically meaningful adversarial examples close to the human perceptual boundary.

Proposed method

• Train a separate VAE p(x|y) for each class to learn class-conditional data distributions.
• Perform gradient-descent in latent space to compute an instance-specific lower bound on log-likelihood for each class (ell_y*).
• Classify by a class-conditioned softmax over exp(alpha ell_y*) with a small offset eta to avoid overconfident decisions on out-of-distribution inputs.
• Optionally apply pixel binarization and discriminative finetuning (Binary ABS) to boost robustness and accuracy.
• Develop and apply diverse adversarial attacks (gradient-based, score-based, decision-based, and transfer-based) including a novel Latent Descent attack and a new L0 decision-based attack.
• Provide analytical bounds (epsilon) on robustness and compare against Madry-style adversarial training.

Experimental results

Research questions

• RQ1 Can an analysis-by-synthesis model using class-conditional data distributions improve adversarial robustness on MNIST beyond state-of-the-art defenses?
• RQ2 How do instance-specific robustness guarantees behave under multiple adversarial threat models (L0, L2, L∞)?
• RQ3 Do adversarial examples for ABS carry semantic meaning and align with human perception, compared to prior defenses?
• RQ4 What is the impact of binarization and discriminative finetuning on robustness and accuracy?

Key findings

• ABS achieves state-of-the-art robustness on MNIST across L0, L2, and L∞ perturbations.
• For L2, ABS outperforms all other models in median adversarial distance.
• For L∞, Binary ABS is state-of-the-art in median perturbation size, with thresholding affecting perceived robustness.
• For L0, ABS and Binary ABS are more robust than all other models, including Madry et al.’s defense.
• Lower-bound robustness estimates indicate ABS μ-L2 robustness ε ≈ 0.690 ± 0.005 (mean) and Binary ABS ≈ 0.601 ± 0.005 under L2 (mean, MNIST).
• Adversarial examples for ABS are semantically meaningful and sit near the perceptual boundary between original and adversarial classes.
• The authors provide extensive attack evaluations (latents, decision-based, transfer-based, gradient-based) and acknowledge robustness evaluation limitations.