[Paper Review] Certified Adversarial Robustness with Additive Noise

Bai Li, Changyou Chen+2 more|arXiv (Cornell University)|Sep 10, 2018

Adversarial Robustness in Machine LearningComputer Science48 references45 citations

TL;DR

The paper links adversarial robustness to robustness against additive Gaussian noise and proposes a scalable certified defense that adds noise at test time and uses stability training to tighten robustness bounds.

ABSTRACT

The existence of adversarial data examples has drawn significant attention in the deep-learning community; such data are seemingly minimally perturbed relative to the original data, but lead to very different outputs from a deep-learning algorithm. Although a significant body of work on developing defensive models has been considered, most such models are heuristic and are often vulnerable to adaptive attacks. Defensive methods that provide theoretical robustness guarantees have been studied intensively, yet most fail to obtain non-trivial robustness when a large-scale model and data are present. To address these limitations, we introduce a framework that is scalable and provides certified bounds on the norm of the input manipulation for constructing adversarial examples. We establish a connection between robustness against adversarial perturbation and additive random noise, and propose a training strategy that can significantly improve the certified bounds. Our evaluation on MNIST, CIFAR-10 and ImageNet suggests that the proposed method is scalable to complicated models and large data sets, while providing competitive robustness to state-of-the-art provable defense methods.

Motivation & Objective

• Motivate and formalize a scalable approach to certified robustness for deep networks under norm-bounded adversarial perturbations.
• Establish a connection between adversarial robustness and robustness to additive random noise via Rényi divergence.
• Develop a training strategy that improves the certified robustness bounds without sacrificing natural accuracy.
• Provide empirical evaluation on MNIST, CIFAR-10, and ImageNet showing competitive provable and empirical robustness.

Proposed method

• Introduce a stochastic classifier by adding Gaussian noise to inputs during testing and derive a certified robustness bound for l2 perturbations using Rényi divergence.
• Provide Algorithm 1 (Certified Robust Classifier) to compute an upper bound L on perturbation size that preserves the predicted class with high probability.
• Prove bounds using Lemma 1 and Theorem 2 (and Theorem 3 for l1 with Laplacian noise) to connect output distributions under noise to adversarial robustness.
• Enhance robustness bounds via Stability Training with Noise (STN), which regularizes the model to be stable under Gaussian perturbations.
• Optionally employ Adversarial Logit Pairing and stability aims to improve accuracy under noise without relying on gradient masking.
• Experiment with stability training and noise parameters on MNIST, CIFAR-10, and ImageNet to compare against PixelDP and TRADES.

Experimental results

Research questions

• RQ1 How large can an adversarial perturbation be while preserving the correct class under randomized smoothing with Gaussian noise?
• RQ2 Can adding noise at test time, combined with stability training, provide certified robustness bounds that scale to large networks and datasets?
• RQ3 How do the derived bounds compare to existing provable defenses (e.g., LP-based or differential privacy-based methods) in both theory and practice?
• RQ4 What is the impact of the noise level on natural accuracy and robustness under various attack strengths?

Key findings

• A certified bound L for l2 perturbations is derived for any classifier with general activation structures when Gaussian noise is added at test time.
• The bound improves when there is a larger gap between the top two class probabilities p(1) and p(2) under noise, and when noise level sigma is optimized.
• Stability Training with Noise (STN) significantly improves the certified bounds and empirical robustness without a large computational burden.
• Empirical results on MNIST, CIFAR-10, and ImageNet show competitive performance against state-of-the-art provable defenses and robustness under strong attacks.
• STN maintains higher natural accuracy than some provable defenses under stronger attacks and achieves competitive robust accuracy.
• The framework demonstrates a scalable approach that can be integrated with existing models and evaluated against adaptive attacks.