[Paper Review] Adversarial vulnerability for any classifier

Alhussein Fawzi, Hamza Fawzi+1 more|arXiv (Cornell University)|Feb 23, 2018

Adversarial Robustness in Machine Learning25 references50 citations

TL;DR

The paper derives classifier-agnostic upper bounds on robustness under smooth generative data models, proves transferability of adversarial perturbations, links in-distribution and unconstrained robustness, and validates bounds with SVHN and CIFAR-10 experiments.

ABSTRACT

Despite achieving impressive performance, state-of-the-art classifiers remain highly vulnerable to small, imperceptible, adversarial perturbations. This vulnerability has proven empirically to be very intricate to address. In this paper, we study the phenomenon of adversarial perturbations under the assumption that the data is generated with a smooth generative model. We derive fundamental upper bounds on the robustness to perturbations of any classification function, and prove the existence of adversarial perturbations that transfer well across different classifiers with small risk. Our analysis of the robustness also provides insights onto key properties of generative models, such as their smoothness and dimensionality of latent space. We conclude with numerical experimental results showing that our bounds provide informative baselines to the maximal achievable robustness on several datasets.

Motivation & Objective

• Motivate robustness limits for any classifier when data is generated by a smooth map g from latent space to images.
• Derive probabilistic bounds on how small perturbations can fool classifiers across arbitrary decision rules.
• Explore transferability of adversarial perturbations and the relation between in-distribution and unconstrained robustness.
• Provide experimental baselines on SVHN and CIFAR-10 to quantify the bounds and guide robust model design.

Proposed method

• Define in-distribution robustness r_in and unconstrained robustness r_unc for a classifier f.
• Model data via a smooth generator g: Z -> X with z ~ N(0, I_d) and x = g(z); impose a modulus of continuity omega: ||g(z) - g(z')|| <= omega(||z - z'||_2).
• Apply Gaussian isoperimetric inequality to derive lower bounds on the probability that r_in is small (Theorem 1).
• Relate r_unc and r_in (Theorem 2) by constructing a nearest-neighbor-based classifier f̃ achieving r_unc >= r_in/2.
• Establish transferability bounds: if two classifiers have small joint risk, there exist common perturbations fooling both (Theorem 3).
• Extend results to approximate generators under 1-Wasserstein distance and derive corresponding robustness bounds (Theorem 4).

Experimental results

Research questions

• RQ1 What are the fundamental limits on robustness to perturbations for any classifier when data is generated by a smooth generator g?
• RQ2 Do adversarial perturbations transfer across different classifiers under the same data-generating model?
• RQ3 How do in-distribution robustness and unconstrained robustness relate, and can one be inferred from the other?
• RQ4 How do bounds behave as the number of classes grows and when the generator approximates the true data distribution?

Key findings

• Upper bounds quantify how easily small perturbations can fool any classifier under smooth, high-dimensional data generation.
• Adversarial perturbations can transfer across classifiers; a small joint risk implies shared perturbations exist (transferability).
• The in-distribution robustness and unconstrained robustness are tightly linked; a simple nearest-neighbor construction guarantees r_unc >= r_in/2.
• Bounds predict increasing fooling probability with more classes and provide realistic baselines for SVHN and CIFAR-10 experiments.
• Experiments with SVHN/CIFAR-10 show the bounds yield informative baselines for maximal achievable robustness and highlight implications for generator smoothness and latent-space dimensionality.