[Paper Review] First-order Adversarial Vulnerability of Neural Networks and Input Dimension

Carl-Johann Simon-Gabriel, Yann Ollivier+3 more|arXiv (Cornell University)|Feb 5, 2018

Adversarial Robustness in Machine Learning62 citations

TL;DR

The paper shows adversarial vulnerability scales with input dimension as the square root of dimension at initialization and remains largely first-order, linking gradient norms to vulnerability across architectures and training regimes.

ABSTRACT

Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions. We show that adversarial vulnerability increases with the gradients of the training objective when viewed as a function of the inputs. Surprisingly, vulnerability does not depend on network topology: for many standard network architectures, we prove that at initialization, the $\ell_1$-norm of these gradients grows as the square root of the input dimension, leaving the networks increasingly vulnerable with growing image size. We empirically show that this dimension dependence persists after either usual or robust training, but gets attenuated with higher regularization.

Motivation & Objective

• Explain why adversarial vulnerability increases with input dimension independent of network topology.
• Link adversarial damage to loss gradients and first-order approximations.
• Evaluate how training (usual and robust) affects dimension-dependent vulnerability.
• Propose and relate gradient-regularization and adversarial augmentation as defenses.
• Assess empirical validity of the first-order model across architectures and training settings.

Proposed method

• Relate adversarial damage to first-order gradient norms via a Taylor expansion of the loss under攻击 perturbations.
• Prove that at initialization, the dual-norm of loss-gradients grows as a function of input dimension, yielding sqrt(d) vulnerability.
• Show that gradient norms are largely independent of network topology under He initialization across common architectures.
• Demonstrate a duality between adversarial augmentation (FGSM/PGD) and gradient regularization (double-backpropagation) as first-order approximations.
• Empirically validate predictions on CIFAR-10 with various regularization strengths and attack norms.
• Discuss the role of regularization (e.g., PGD-regularization) in dampening dimension dependence.

Experimental results

Research questions

• RQ1 Does adversarial vulnerability depend on input dimension independent of network topology?
• RQ2 How are gradient norms of the loss with respect to inputs related to adversarial vulnerability across architectures?
• RQ3 Can gradient regularization or adversarial augmentation bridge the vulnerability-dimension trade-off, and how do they compare?
• RQ4 How does training (usual vs robust) affect the dimension-dependent growth of gradient norms and vulnerability?

Key findings

• At initialization, the L2/Lq gradient norms with respect to inputs scale with dimension, causing sqrt(d) growth in adversarial vulnerability for common nets.
• The gradient-based vulnerability largely explains adversarial damage, with a strong first-order relationship between gradient norms and vulnerability.
• Adversarial augmentation and gradient regularization follow a dual relationship, yielding similar accuracy-vulnerability trade-offs and supporting first-order dominance.
• PGD-regularization during training significantly dampens the dimension dependence of vulnerability without severely harming accuracy, more so with higher regularization.
• The gradient norms on training vs. test sets differ, suggesting gradient properties do not generalize well beyond training data.