[Paper Review] Enhancing the Transferability of Adversarial Attacks through Variance Tuning

Xiaosen Wang, Kun He|arXiv (Cornell University)|Mar 29, 2021

Adversarial Robustness in Machine LearningComputer Science36 references26 citations

TL;DR

The paper introduces variance tuning to gradient-based iterative adversarial attacks to boost transferability, especially against black-box defenses, while preserving white-box success. It shows strong gains on ImageNet and with input transformations and ensembles.

ABSTRACT

Deep neural networks are vulnerable to adversarial examples that mislead the models with imperceptible perturbations. Though adversarial attacks have achieved incredible success rates in the white-box setting, most existing adversaries often exhibit weak transferability in the black-box setting, especially under the scenario of attacking models with defense mechanisms. In this work, we propose a new method called variance tuning to enhance the class of iterative gradient based attack methods and improve their attack transferability. Specifically, at each iteration for the gradient calculation, instead of directly using the current gradient for the momentum accumulation, we further consider the gradient variance of the previous iteration to tune the current gradient so as to stabilize the update direction and escape from poor local optima. Empirical results on the standard ImageNet dataset demonstrate that our method could significantly improve the transferability of gradient-based adversarial attacks. Besides, our method could be used to attack ensemble models or be integrated with various input transformations. Incorporating variance tuning with input transformations on iterative gradient-based attacks in the multi-model setting, the integrated method could achieve an average success rate of 90.1% against nine advanced defense methods, improving the current best attack performance significantly by 85.1% . Code is available at https://github.com/JHL-HUST/VT.

Motivation & Objective

• Motivate and improve the transferability of gradient-based adversarial examples, particularly against defended (black-box) models.
• Propose a variance-tuning mechanism to stabilize gradient updates across iterations.
• Demonstrate that variance tuning enhances attacks when combined with input transformations and ensemble models.
• Show that the proposed method maintains white-box performance while significantly increasing black-box success rates.

Proposed method

• Define gradient variance in a neighborhood around the current input.
• At each iteration, tune the current gradient using the neighborhood variance prior to the momentum step.
• Update the adversarial example via a sign of the variance-tuned gradient with a fixed step size.
• Extend the approach to NI-FGSM and DIM/TIM/SIM input transformations (CTM).
• Use sampling (N) in the neighborhood to approximate gradient variance and a neighborhood bound (beta).
• Evaluate on ImageNet against normally trained and adversarially trained models, including nine defense models.

Experimental results

Research questions

• RQ1 Can variance tuning improve the transferability of iterative gradient-based attacks to unseen models or defended models?
• RQ2 How do hyperparameters of variance tuning (neighborhood size beta and sample count N) affect transferability and computation?

Key findings

• Variance-tuned attacks (VMI-FGSM/VNI-FGSM) outperform baseline MI-FGSM/NI-FGSM on black-box models while keeping white-box performance high.
• Integrating variance tuning with input transformations (DIM/TIM/SIM) further boosts transferability, especially against adversarially trained models.
• In ensemble settings, variance-tuned attacks achieve substantial gains over baselines, with notable improvements against advanced defenses.
• On nine defense models, the integrated method attains an average success rate of 90.1% in the multi-model setting, outperforming baselines by 6.6%.
• Ablation analyses show beta around 1.5 and N around 20 balance transferability and computation costs.