[Paper Review] A Boundary Tilting Persepective on the Phenomenon of Adversarial Examples

Thomas Tanay, Lewis D. Griffin|arXiv (Cornell University)|Aug 27, 2016

Adversarial Robustness in Machine Learning9 references136 citations

TL;DR

The paper critiques the linear explanation of adversarial examples and introduces a boundary tilting framework showing how adversarial strength depends on the deviation from the nearest centroid boundary and data submanifolds. It also links adversarial strength to regularisation and provides a linear-class analysis with SVM experiments.

ABSTRACT

Deep neural networks have been shown to suffer from a surprising weakness: their classification outputs can be changed by small, non-random perturbations of their inputs. This adversarial example phenomenon has been explained as originating from deep networks being "too linear" (Goodfellow et al., 2014). We show here that the linear explanation of adversarial examples presents a number of limitations: the formal argument is not convincing, linear classifiers do not always suffer from the phenomenon, and when they do their adversarial examples are different from the ones affecting deep networks. We propose a new perspective on the phenomenon. We argue that adversarial examples exist when the classification boundary lies close to the submanifold of sampled data, and present a mathematical analysis of this new perspective in the linear case. We define the notion of adversarial strength and show that it can be reduced to the deviation angle between the classifier considered and the nearest centroid classifier. Then, we show that the adversarial strength can be made arbitrarily high independently of the classification performance due to a mechanism that we call boundary tilting. This result leads us to defining a new taxonomy of adversarial examples. Finally, we show that the adversarial strength observed in practice is directly dependent on the level of regularisation used and the strongest adversarial examples, symptomatic of overfitting, can be avoided by using a proper level of regularisation.

Motivation & Objective

• Motivate a shift from purely linear explanations of adversarial examples to a boundary tilting perspective.
• Characterise when adversarial examples exist by considering the alignment between decision boundaries and data submanifolds.
• Quantify adversarial strength in linear models and relate it to deviation from the nearest centroid classifier.
• Investigate how boundary tilting and regularisation influence adversarial strength and model robustness.
• Propose a taxonomy of adversarial examples based on boundary geometry and data distribution.

Proposed method

• Define a strict non-existence condition for adversarial examples using a linear classifier boundary and mirror images.
• Introduce the strength measure s(I, C) = arctan(||j - m(i, C)|| / ||i - m(i, C)||) and show it reduces to the deviation angle delta_c between C and the nearest-centroid boundary B.
• Express classifier deviation as c = cos(delta_c) b + sin(delta_c) b_perp_c and derive s(I, C) and s(J, C) formulas depending on delta_c and rc = c0/||i||.
• Demonstrate that boundary tilting can yield arbitrarily strong adversarial examples without harming performance, via analysis of rc and delta_c.
• Show that with high regularisation, adversarial strength shrinks toward the nearest centroid classifier; low regularisation increases overfitting and boundary tilting.
• Provide experimental intuition using SVM to relate observed adversarial strength to regularisation levels.

Experimental results

Research questions

• RQ1 Under what geometric conditions do adversarial examples exist when the data lie on a submanifold and the decision boundary lies near it?
• RQ2 How can adversarial strength be quantified in linear models, and what role does the deviation from the nearest centroid boundary play?
• RQ3 Can boundary tilting cause strong adversarial examples without sacrificing classification accuracy, and how does regularisation modulate this effect?
• RQ4 What is the relationship between boundary geometry (deviation angle) and the strength of adversarial examples across data distributions?

Key findings

• Adversarial strength can be characterized by the deviation angle delta_c between a classifier’s boundary and the nearest centroid boundary.
• When the boundary tilts along directions with low data variance, adversarial strength can become arbitrarily large without degrading classification performance.
• The strength measure simplifies to s ≈ |delta_c| when rc ≈ 0, linking robustness directly to boundary alignment with data means.
• Stronger regularisation reduces adversarial strength, pushing the classifier toward the nearest centroid, while weak regularisation fosters boundary tilting and stronger adversarial examples.
• Experiments with linear models (SVM) suggest practical control of adversarial strength through regularisation, contrasting with deeper networks where perturbations can be imperceptible.