[Paper Review] Excessive Invariance Causes Adversarial Vulnerability

Jörn-Henrik Jacobsen, Jens Behrmann+2 more|arXiv (Cornell University)|Nov 1, 2018

Adversarial Robustness in Machine LearningComputer Science50 references59 citations

TL;DR

The paper argues that deep networks are not only too sensitive to irrelevant changes but also overly invariant to task-relevant changes, causing wide-ranging adversarial vulnerabilities. It introduces invertible networks to study this and proposes an information-theoretic objective to reduce excessive invariance.

ABSTRACT

Despite their impressive performance, deep neural networks exhibit striking failures on out-of-distribution inputs. One core idea of adversarial example research is to reveal neural network errors under such distribution shifts. We decompose these errors into two complementary sources: sensitivity and invariance. We show deep networks are not only too sensitive to task-irrelevant changes of their input, as is well-known from epsilon-adversarial examples, but are also too invariant to a wide range of task-relevant changes, thus making vast regions in input space vulnerable to adversarial attacks. We show such excessive invariance occurs across various tasks and architecture types. On MNIST and ImageNet one can manipulate the class-specific content of almost any image without changing the hidden activations. We identify an insufficiency of the standard cross-entropy loss as a reason for these failures. Further, we extend this objective based on an information-theoretic analysis so it encourages the model to consider all task-dependent features in its decision. This provides the first approach tailored explicitly to overcome excessive invariance and resulting vulnerabilities.

Motivation & Objective

• Identify excessive invariance as a core cause of adversarial vulnerability across tasks and architectures.
• Demonstrate that class-specific content can be altered without changing activations (invariance-based adversarial examples).
• Provide a method to access and manipulate decision-relevant and nuisance factors using invertible networks.
• Link cross-entropy’s limitations to information-theoretic properties and propose a remedy.
• Show practical improvements using the independence cross-entropy objective on benchmark datasets.

Proposed method

• Define pre-images and connect invariance to adversarial examples.
• Use fully invertible RevNets to access semantic (logits) and nuisance (hidden) variables.
• Develop metameric sampling to visualize and analyze dependencies between z_s and z_n.
• Identify insufficiency of cross-entropy via information-theoretic analysis.
• Propose independence cross-entropy (iCE) loss with nuisance classifier and an optional maximum-likelihood term for nuisance factors.
• Evaluate iCE on MNIST, ImageNet, and shiftMNIST variants, including an adversarial-spheres toy problem.

Experimental results

Research questions

• RQ1 Can classifiers be excessively invariant to task-relevant changes, leading to adversarial vulnerability?
• RQ2 Do invertible networks reveal and quantify invariant subspaces that standard architectures ignore?
• RQ3 Can an information-theoretic objective promote learning of all task-dependent features to reduce invariances?
• RQ4 Does the independence cross-entropy objective mitigate invariance-based adversarial vulnerabilities in practice?

Key findings

• Invariance-based adversarial examples exist across MNIST, ImageNet, and synthetic tasks, where class content can be altered without changing activations.
• Fully invertible RevNets reveal that logits can remain constant while semantic content changes dramatically, indicating excessive invariance.
• Metameric sampling shows nuisance variables can dominate appearance while keeping logit activations unchanged.
• Cross-entropy alone may maximize mutual information between labels and representations, but under distribution shifts this can reduce robustness; iCE prevents this by minimizing information about nuisances.
• The independence cross-entropy objective reduces invariance-based vulnerabilities and improves robustness under distribution shifts (e.g., shiftMNIST) by up to ~38 percentage points in some setups.
• On ILSVRC-2012 validation, fi-RevNet variants achieve competitive Top1/Top5 errors compared to VGG19 and ResNets (e.g., fi-RevNet48: Top1 29.50, Top5 11.30).