[Paper Review] CAAD 2018: Generating Transferable Adversarial Examples

Yash Sharma, Tiendung Le+1 more|arXiv (Cornell University)|Sep 29, 2018

Adversarial Robustness in Machine Learning27 references4 citations

TL;DR

This paper presents a state-of-the-art method for generating transferable adversarial examples in the CAAD 2018 competition, leveraging spatial gradient smoothing and randomized preprocessing to achieve high transferability across unknown models. The authors won 1st place in both non-targeted and targeted attack tracks, demonstrating that gradient-based optimization with randomization and smoothing significantly improves transferability while meeting strict computational constraints.

ABSTRACT

Deep neural networks (DNNs) are vulnerable to adversarial examples, perturbations carefully crafted to fool the targeted DNN, in both the non-targeted and targeted case. In the non-targeted case, the attacker simply aims to induce misclassification. In the targeted case, the attacker aims to induce classification to a specified target class. In addition, it has been observed that strong adversarial examples can transfer to unknown models, yielding a serious security concern. The NIPS 2017 competition was organized to accelerate research in adversarial attacks and defenses, taking place in the realistic setting where submitted adversarial attacks attempt to transfer to submitted defenses. The CAAD 2018 competition took place with nearly identical rules to the NIPS 2017 one. Given the requirement that the NIPS 2017 submissions were to be open-sourced, participants in the CAAD 2018 competition were able to directly build upon previous solutions, and thus improve the state-of-the-art in this setting. Our team participated in the CAAD 2018 competition, and won 1st place in both attack subtracks, non-targeted and targeted adversarial attacks, and 3rd place in defense. We outline our solutions and development results in this article. We hope our results can inform researchers in both generating and defending against adversarial examples.

Motivation & Objective

• To develop highly transferable adversarial examples that successfully fool unknown models in a realistic, black-box setting.
• To improve upon prior adversarial attack methods by enhancing transferability while adhering to strict computational and perturbation constraints.
• To design a robust defense mechanism that maintains high clean accuracy while resisting strong adversarial attacks.
• To investigate the effectiveness of gradient smoothing and randomization in improving transferability and robustness.
• To submit a defense that performs well against strong, previously unseen attacks, particularly those exploiting model-specific vulnerabilities.

Proposed method

• The non-targeted attack uses a trained Adversarial Transformation Network (ATN) to perform offline optimization, bypassing real-time constraints.
• For targeted attacks, the method combines gradient-based optimization with randomized preprocessing to enhance transferability, though initially too slow for the time limit.
• Spatial gradient smoothing is introduced as an efficient alternative that maintains high transferability while meeting computational constraints.
• The final solution combines gradient smoothing with iterative randomization to further improve transferability within the time limit.
• The defense employs a '2 MSB' technique, passing only the two most significant bits of input images to an ensemble of adversarially trained models to reduce sensitivity to perturbations.
• Additional robustness is achieved by adding mild Bernoulli noise and a class-patch mechanism that avoids misclassifying as the 'Jigsaw puzzle' class when the top prediction is that class.

Experimental results

Research questions

• RQ1 Can gradient smoothing be used to generate transferable adversarial examples efficiently under strict time constraints?
• RQ2 How does combining randomization with gradient-based optimization affect transferability in targeted attacks?
• RQ3 To what extent can a defense based on bit-level input manipulation and noise injection improve robustness against strong, adaptive attacks?
• RQ4 Why does gradient smoothing exhibit such strong transferability despite not being explicitly designed for it?
• RQ5 Can a defense be made robust against a specific, known attack (e.g., ATN) by incorporating knowledge of the attack's behavior?

Key findings

• The combination of gradient smoothing and randomization achieved 1st place in both non-targeted and targeted attack subtracks, demonstrating superior transferability.
• The '2 MSB' defense achieved 89.3% clean accuracy and 42.0% robustness against the ATN attack when enhanced with noise and class-patching.
• The 'Noise + Avoid' defense variant improved robustness against the ATN attack from 2.0% to 42.0%, showing that model-specific knowledge can significantly enhance defense performance.
• Gradient smoothing alone achieved better transferability than pure randomization, with 'Grad + Rand' outperforming 'Rand' significantly in both white-box and transfer settings.
• The 'Dropout' defense, though more aggressive and less accurate on clean data, outperformed '2 MSB' against the ATN attack, highlighting the trade-off between clean accuracy and robustness.
• The final defense solution, '2 MSB' with noise and avoidance, was selected for submission due to its balance of clean accuracy and robustness, especially against the team’s own attack.