[Paper Review] Stochastic Activation Pruning for Robust Adversarial Defense

Guneet S. Dhillon, Kamyar Azizzadenesheli+5 more|arXiv (Cornell University)|Mar 5, 2018

Adversarial Robustness in Machine LearningComputer Science20 references206 citations

TL;DR

SAP applies stochastic activation pruning to pretrained networks as a post-hoc defense against adversarial examples, improving robustness and calibration without fine-tuning, and yields additive benefits with adversarial training.

ABSTRACT

Neural networks are known to be vulnerable to adversarial examples. Carefully chosen perturbations to real images, while imperceptible to humans, induce misclassification and threaten the reliability of deep learning systems in the wild. To guard against adversarial examples, we take inspiration from game theory and cast the problem as a minimax zero-sum game between the adversary and the model. In general, for such games, the optimal strategy for both players requires a stochastic policy, also known as a mixed strategy. In this light, we propose Stochastic Activation Pruning (SAP), a mixed strategy for adversarial defense. SAP prunes a random subset of activations (preferentially pruning those with smaller magnitude) and scales up the survivors to compensate. We can apply SAP to pretrained networks, including adversarially trained models, without fine-tuning, providing robustness against adversarial examples. Experiments demonstrate that SAP confers robustness against attacks, increasing accuracy and preserving calibration.

Motivation & Objective

• Motivate robustness to adversarial examples in neural networks using a game-theoretic viewpoint.
• Introduce SAP as a stochastic mixed strategy to defend pretrained models post-hoc.
• Demonstrate SAP’s effectiveness against adversarial perturbations and assess calibration.
• Compare SAP to dropout and adversarial training, and explore reinforcement learning applicability.

Proposed method

• Define SAP as a mixed strategy in a minimax game between adversary and defender.
• Prune a random subset of activations in each layer by sampling from a multinomial distribution proportional to activation magnitudes.
• Scale surviving activations to preserve the dynamic range via inverse sampling probability, preserving expected activations.
• Apply SAP post-hoc to pretrained networks without fine-tuning.
• Evaluate SAP on image classification (CIFAR-10 with ResNet-20) and deep RL (DDQN on Atari).
• Compare SAP to dropout, Gaussian/noise baselines, and adversarial training, using MC sampling to estimate gradients for stochastic models.

Experimental results

Research questions

• RQ1 Can SAP improve robustness of pretrained networks to adversarial perturbations without fine-tuning?
• RQ2 How does SAP affect accuracy, calibration, and robustness under FGSM and iterative attacks across vision and reinforcement learning tasks?
• RQ3 How does SAP interact with adversarial training and other stochastic defenses?

Key findings

• SAP-backed models show improved accuracy against adversarial perturbations at certain perturbation levels (e.g., SAP-100 yields absolute gains at lambda values like 1, 2, 4 in CIFAR-10 experiments).
• SAP preserves accuracy for moderate perturbations and improves calibration relative to the dense model.
• Adversarial training combined with SAP (ADV + SAP-100) achieves higher accuracy than adversarial training alone at larger perturbation magnitudes.
• In reinforcement learning, SAP-100 yields substantial relative reward improvements across several Atari games for nonzero perturbations, with some cases showing very large gains.
• SAP tends to outperform dropout as a stochastic defense and remains effective as a post-hoc modification without retraining.