[Paper Review] On the Geometry of Adversarial Examples

Marc Khoury, Dylan Hadfield-Menell|arXiv (Cornell University)|Nov 1, 2018

Adversarial Robustness in Machine LearningComputer Science35 references43 citations

TL;DR

The paper introduces a geometric framework for adversarial examples based on data manifolds in high dimensions, highlighting codimension as a key factor, proving norm-based robustness tradeoffs, and showing sampling efficiency gaps between nearest-neighbor and ball-based methods.

ABSTRACT

Adversarial examples are a pervasive phenomenon of machine learning models where seemingly imperceptible perturbations to the input lead to misclassifications for otherwise statistically accurate models. We propose a geometric framework, drawing on tools from the manifold reconstruction literature, to analyze the high-dimensional geometry of adversarial examples. In particular, we highlight the importance of codimension: for low-dimensional data manifolds embedded in high-dimensional space there are many directions off the manifold in which to construct adversarial examples. Adversarial examples are a natural consequence of learning a decision boundary that classifies the low-dimensional data manifold well, but classifies points near the manifold incorrectly. Using our geometric framework we prove (1) a tradeoff between robustness under different norms, (2) that adversarial training in balls around the data is sample inefficient, and (3) sufficient sampling conditions under which nearest neighbor classifiers and ball-based adversarial training are robust.

Motivation & Objective

• Formalize the manifold-based model of data and adversarial perturbations.
• Quantify how high codimension affects robustness to adversarial perturbations.
• Show limitations of ball-based adversarial training and sampling requirements.
• Propose and analyze robust classification strategies, including nearest-neighbor methods.

Proposed method

• Model data as samples from class-specific low-dimensional manifolds embedded in high-dimensional space.
• Define the decision axis as a generalization of maximum margin that accounts for manifold curvature.
• Introduce the concept of epsilon-tubular neighborhoods around the manifold to represent adversarial examples.
• Prove a norm-based tradeoff showing Lambda_2 and Lambda_infty differ, implying norm-specific robustness limits.
• Provide sampling-based guarantees for nearest-neighbor and ball-based learning methods under epsilon-robustness.
• Use synthetic and MNIST experiments to validate theoretical results.

Experimental results

Research questions

• RQ1 How does the codimension between data manifolds and the ambient space affect adversarial robustness?
• RQ2 Can a single decision boundary be robust to perturbations across different norms, or are norm-specific tradeoffs inevitable?
• RQ3 Are ball-based adversarial training methods sample-efficient compared to nearest-neighbor classifiers in achieving robustness?
• RQ4 Do nearest-neighbor classifiers offer robustness advantages in high codimension settings?

Key findings

• There exists a tradeoff in robustness between norms, i.e., Lambda_2 and Lambda_infty are generally distinct, leading to norm-specific robustness limits.
• Adversarial training on balls around training data is sample inefficient for achieving robust classification.
• Nearest neighbor classifiers can be robust in high codimension given sufficient sampling, owing to elongated Voronoi cells.
• X^epsilon (ball-based augmentation) often poorly models M^epsilon, explaining limited robustness gains from standard adversarial training.
• For certain constructions, nearest-neighbor methods require exponentially fewer samples than the learning algorithm L to achieve the same robustness.
• Experimental results on synthetic data and MNIST support the theoretical claims about norm-specific robustness and sampling efficiency.