[Paper Review] Sampling-based Fast Gradient Rescaling Method for Highly Transferable Adversarial Attacks

Xu Han, Anmin Liu+3 more|arXiv (Cornell University)|Jul 6, 2023

Adversarial Robustness in Machine LearningComputer Science3 citations

TL;DR

This paper proposes a Sampling-based Fast Gradient Rescaling Method (S-FGRM) that replaces the sign function in gradient-based adversarial attacks with a data rescaling technique and a Depth First Sampling strategy to improve gradient approximation accuracy and boost adversarial transferability. The method significantly outperforms state-of-the-art baselines on ImageNet, achieving higher black-box attack success rates with minimal computational overhead.

ABSTRACT

Deep neural networks are known to be vulnerable to adversarial examples crafted by adding human-imperceptible perturbations to the benign input. After achieving nearly 100% attack success rates in white-box setting, more focus is shifted to black-box attacks, of which the transferability of adversarial examples has gained significant attention. In either case, the common gradient-based methods generally use the sign function to generate perturbations on the gradient update, that offers a roughly correct direction and has gained great success. But little work pays attention to its possible limitation. In this work, we observe that the deviation between the original gradient and the generated noise may lead to inaccurate gradient update estimation and suboptimal solutions for adversarial transferability. To this end, we propose a Sampling-based Fast Gradient Rescaling Method (S-FGRM). Specifically, we use data rescaling to substitute the sign function without extra computational cost. We further propose a Depth First Sampling method to eliminate the fluctuation of rescaling and stabilize the gradient update. Our method could be used in any gradient-based attacks and is extensible to be integrated with various input transformation or ensemble methods to further improve the adversarial transferability. Extensive experiments on the standard ImageNet dataset show that our method could significantly boost the transferability of gradient-based attacks and outperform the state-of-the-art baselines.

Motivation & Objective

• To address the limitation of the sign function in gradient-based adversarial attacks, which provides a coarse approximation of true gradients and may mislead optimization paths.
• To improve the accuracy of gradient update direction estimation by replacing the sign function with a data rescaling method that preserves gradient magnitude disparities.
• To stabilize gradient updates by eliminating local fluctuation errors through a novel Depth First Sampling strategy.
• To develop a general-purpose method compatible with any gradient-based attack, extensible to input transformations and model ensembles for further transferability gains.

Proposed method

• Replace the sign function in standard FGSM-style attacks with a data rescaling function that maps gradient values to [-1, 1] based on their distribution, preserving relative magnitude differences.
• Introduce a Depth First Sampling (DFS) method that selects gradient components in a depth-first traversal order to reduce local fluctuation and stabilize the update direction.
• Apply the rescaling function to the gradient before perturbation generation, ensuring more accurate and consistent gradient direction estimation.
• Integrate the S-FGRM method into existing attacks such as MI-FGSM and NI-FGSM, enabling it to be used with input transformations or ensemble methods.
• Use a configurable parameter (c) in the rescaling function to control the scaling intensity, with ablation studies showing optimal performance at c=2.
• The method maintains computational efficiency, requiring only O(S) time per iteration (vs. O(S log S) for sorting-based alternatives like Staircase).

Experimental results

Research questions

• RQ1 Does replacing the sign function with a data rescaling method lead to more accurate gradient approximation and improved adversarial transferability?
• RQ2 Can the Depth First Sampling strategy effectively reduce local fluctuation in gradient updates and stabilize the optimization path?
• RQ3 How does S-FGRM compare to state-of-the-art methods like Staircase (S²M) in terms of attack success rate and computational efficiency?
• RQ4 To what extent can S-FGRM be generalized and extended when combined with input transformations or model ensembles?
• RQ5 Does the proposed method mitigate overfitting in gradient-based attacks and improve performance in black-box settings?

Key findings

• S-FGRM significantly improves transferability on ImageNet, achieving 99.3% attack success rate on Inc-v3 and 86.2% on Inc-v4 in the single-model setting with input transformation.
• The method outperforms both MI-FGSM and NI-FGSM baselines, with SMI-CT-FGSM achieving 99.3% on Inc-v3 and 86.2% on Inc-v4, surpassing sgMI-CT-FGSM by 0.1–0.3%.
• DFS sampling achieves better transferability with fewer samples (N=12) than baseline methods, demonstrating its efficiency and stability.
• S-FGRM achieves comparable performance to the Staircase method (S²M) but with lower computational cost—O(S) vs. O(S log S) per iteration—making it more scalable.
• When integrated with ensemble attacks, S-FGRM consistently outperforms baselines across multiple target models, including Inc-v3, Inc-v4, IncRes-v2, and Res-101.
• The ablation study confirms that rescaling and DFS sampling jointly reduce overfitting and improve optimization stability, leading to higher black-box attack success rates.