[论文解读] On the Geometry of Adversarial Examples

Marc Khoury, Dylan Hadfield-Menell|arXiv (Cornell University)|Nov 1, 2018

Adversarial Robustness in Machine Learning参考文献 35被引用 43

一句话总结

本文提出一个基于高维数据流形的对抗样本几何框架，强调协维数是关键因素，证明基于范数的鲁棒性权衡，并展示最近邻与基于球的采样方法在采样效率上的差距。

ABSTRACT

Adversarial examples are a pervasive phenomenon of machine learning models where seemingly imperceptible perturbations to the input lead to misclassifications for otherwise statistically accurate models. We propose a geometric framework, drawing on tools from the manifold reconstruction literature, to analyze the high-dimensional geometry of adversarial examples. In particular, we highlight the importance of codimension: for low-dimensional data manifolds embedded in high-dimensional space there are many directions off the manifold in which to construct adversarial examples. Adversarial examples are a natural consequence of learning a decision boundary that classifies the low-dimensional data manifold well, but classifies points near the manifold incorrectly. Using our geometric framework we prove (1) a tradeoff between robustness under different norms, (2) that adversarial training in balls around the data is sample inefficient, and (3) sufficient sampling conditions under which nearest neighbor classifiers and ball-based adversarial training are robust.

研究动机与目标

• Formalize the manifold-based model of data and adversarial perturbations.
• Quantify how high codimension affects robustness to adversarial perturbations.
• Show limitations of ball-based adversarial training and sampling requirements.
• Propose and analyze robust classification strategies, including nearest-neighbor methods.

提出的方法

• Model data as samples from class-specific low-dimensional manifolds embedded in high-dimensional space.
• Define the decision axis as a generalization of maximum margin that accounts for manifold curvature.
• Introduce the concept of epsilon-tubular neighborhoods around the manifold to represent adversarial examples.
• Prove a norm-based tradeoff showing Lambda_2 and Lambda_infty differ, implying norm-specific robustness limits.
• Provide sampling-based guarantees for nearest-neighbor and ball-based learning methods under epsilon-robustness.
• Use synthetic and MNIST experiments to validate theoretical results.

实验结果

研究问题

• RQ1 How does the codimension between data manifolds and the ambient space affect adversarial robustness?
• RQ2 Can a single decision boundary be robust to perturbations across different norms, or are norm-specific tradeoffs inevitable?
• RQ3 Are ball-based adversarial training methods sample-efficient compared to nearest-neighbor classifiers in achieving robustness?
• RQ4 Do nearest-neighbor classifiers offer robustness advantages in high codimension settings?

主要发现

• There exists a tradeoff in robustness between norms, i.e., Lambda_2 and Lambda_infty are generally distinct, leading to norm-specific robustness limits.
• Adversarial training on balls around training data is sample inefficient for achieving robust classification.
• Nearest neighbor classifiers can be robust in high codimension given sufficient sampling, owing to elongated Voronoi cells.
• X^epsilon (ball-based augmentation) often poorly models M^epsilon, explaining limited robustness gains from standard adversarial training.
• For certain constructions, nearest-neighbor methods require exponentially fewer samples than the learning algorithm L to achieve the same robustness.
• Experimental results on synthetic data and MNIST support the theoretical claims about norm-specific robustness and sampling efficiency.