[Paper Review] Diversity can be Transferred: Output Diversification for White- and Black-box Attacks

Yusuke Tashiro, Yang Song+1 more|arXiv (Cornell University)|Mar 15, 2020

Adversarial Robustness in Machine Learning54 references45 citations

TL;DR

Introduces Output Diversified Sampling (ODS), a gradient-based sampling strategy that maximizes diversity in a model’s outputs to improve both white-box and black-box adversarial attacks, with transferability via surrogate models.

ABSTRACT

Adversarial attacks often involve random perturbations of the inputs drawn from uniform or Gaussian distributions, e.g., to initialize optimization-based white-box attacks or generate update directions in black-box attacks. These simple perturbations, however, could be sub-optimal as they are agnostic to the model being attacked. To improve the efficiency of these attacks, we propose Output Diversified Sampling (ODS), a novel sampling strategy that attempts to maximize diversity in the target model's outputs among the generated samples. While ODS is a gradient-based strategy, the diversity offered by ODS is transferable and can be helpful for both white-box and black-box attacks via surrogate models. Empirically, we demonstrate that ODS significantly improves the performance of existing white-box and black-box attacks. In particular, ODS reduces the number of queries needed for state-of-the-art black-box attacks on ImageNet by a factor of two.

Motivation & Objective

• Motivate the need for more output-diverse perturbations beyond input-space random sampling in adversarial attacks.
• Propose Output Diversified Sampling (ODS) to maximize diversity in the target model’s outputs.
• Show that ODS is transferable via surrogate models and improves both white-box and black-box attack efficiency.
• Demonstrate empirical improvements in attack effectiveness and query efficiency on standard datasets and models.

Proposed method

• Define the ODS perturbation direction v_ODS(x,f,w_d) as the normalized gradient of w_d^T f(x) with respect to x.
• In white-box settings, use ODI (Initialization with ODI) to generate diversified starting points by maximizing w_d^T(f(x)-f(x_org)) within an ε-ball.
• In black-box settings, apply ODS to surrogate models to generate update directions for attacks like SimBA and Boundary Attack.
• Demonstrate that diversity in input perturbations translates to diversity in output space, and that surrogate-model-based ODS transfers this diversity to the target model.
• Compare against naïve random initializations and standard baselines to show improved diversity and efficiency.

Experimental results

Research questions

• RQ1 Does perturbing inputs along outputs-directed gradients yield more diverse and effective adversarial starts than random input-space perturbations?
• RQ2 Can ODS-diversified perturbations created on surrogate models transfer diversity to target black-box models?
• RQ3 How does ODI-augmented white-box ATTACK performance compare to state-of-the-art attacks under ℓ∞ and ℓ2 norms?
• RQ4 Does combining ODS with surrogate-model-based black-box attacks reduce query counts and perturbation sizes?
• RQ5 Is ODS effective when surrogate models are trained on out-of-distribution data?

Key findings

• ODS improves attack effectiveness in white-box settings for both ℓ∞ and ℓ2 attacks.
• ODI with ODS yields more diverse starting points than uniform initialization, reducing required perturbations in CIFAR-10 and ImageNet experiments.
• ODS applied to surrogate models substantially reduces query counts for score-based attacks on ImageNet (about a factor of two).
• ODS improves decision-based and score-based black-box attacks when combined with surrogate models, outperforming several state-of-the-art methods in query-efficiency and perturbation size.
• ODS remains effective even when surrogates are trained on out-of-distribution data, offering robustness to surrogate data quality.