[Paper Review] NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks

Yandong Li, Lijun Li+3 more|arXiv (Cornell University)|May 1, 2019

Adversarial Robustness in Machine Learning109 citations

TL;DR

NATTACK learns a probability distribution over a small region around an input to craft adversarial examples in a black-box setting, beating many defenses and matching white-box methods across diverse DNNs.

ABSTRACT

Powerful adversarial attack methods are vital for understanding how to construct robust deep neural networks (DNNs) and for thoroughly testing defense techniques. In this paper, we propose a black-box adversarial attack algorithm that can defeat both vanilla DNNs and those generated by various defense techniques developed recently. Instead of searching for an "optimal" adversarial example for a benign input to a targeted DNN, our algorithm finds a probability density distribution over a small region centered around the input, such that a sample drawn from this distribution is likely an adversarial example, without the need of accessing the DNN's internal layers or weights. Our approach is universal as it can successfully attack different neural networks by a single algorithm. It is also strong; according to the testing against 2 vanilla DNNs and 13 defended ones, it outperforms state-of-the-art black-box or white-box attack methods for most test cases. Additionally, our results reveal that adversarial training remains one of the best defense techniques, and the adversarial examples are not as transferable across defended DNNs as them across vanilla DNNs.

Motivation & Objective

• Motivate robust testing of DNNs by developing a strong, universal black-box adversarial attack.
• Propose a distribution-based optimization framework that smooths the attack objective and avoids gradient estimation on non-smooth networks.
• Demonstrate effectiveness against vanilla and defended DNNs, and analyze transferability of adversarial examples.
• Show that adversarial training remains a strong defense and that transferability across defended models is limited.

Proposed method

• Formulate adversarial attack as learning a probability distribution over an ellipsoidal region S around the input x.
• Define a transformation g to map latent samples to the input space and project onto S to produce x′.
• Optimize a smoothed objective J(θ)=E[f(projS(g(z)))] w.r.t. distribution parameters θ via NES-inspired updates to the mean μ (and bandwidth σ).
• Use a regression network to initialize μ0 to accelerate convergence.
• Employ a C&W-style loss f(x′)=max(0, log F(x′)y − maxc≠y log F(x′)c) to drive misclassification.
• Integrate projection into the objective by absorbing projS(g(z)) into f, improving stability over gradient-estimate-based methods.
• Compare against white-box and other black-box attacks, including ZOO, QL, and BPDA, across 2 vanilla and 13 defended DNNs.

Experimental results

Research questions

• RQ1 Can a single universal, gradient-free attack defeat both vanilla and defended DNNs without access to internal model weights?
• RQ2 Does learning a distribution over a local input region enable more robust adversarial generation than direct gradient-free optimization?
• RQ3 How does NATTACK perform relative to state-of-the-art black-box and white-box attacks across diverse defenses and datasets?
• RQ4 What is the transferability of adversarial examples between defended models when attacked via a black-box method?

Key findings

• NATTACK achieves high attack strength across 2 vanilla DNNs and 13 defended DNNs, outperforming several state-of-the-art black-box methods and rivaling white-box approaches in many cases.
• On 13 defenses, NATTACK attains 100% attack success on six defenses and >90% on five others under the reported protocol.
• Compared to QL, NATTACK shows improved performance, particularly when ablation studies demonstrate gains from the smoothed objective, the change-of-variable g(·), and z-scoring stabilization.
• When attacking ImageNet, using the regression initialization reduces run-time from ~71s (without regression) to ~48s; on CIFAR-10, attacks take about ~30s for BPDA, with NATTACK showing competitive timing.
• Adversarial training remains one of the strongest defenses, and transferability of adversarial examples across defended DNNs is weaker than across vanilla DNNs, reducing the utility of substitute-model attacks.
• NATTACK provides a scalable framework for generating large sets of adversarial examples by sampling from a learned distribution, potentially aiding robust defense training.