[Paper Review] Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks

Ambra Demontis, Marco Melis+6 more|arXiv (Cornell University)|Sep 8, 2018

Adversarial Robustness in Machine LearningComputer Science53 references93 citations

TL;DR

This paper presents a unified gradient-based framework for evasion and poisoning attacks, formalizes transferability, and identifies factors driving transferability across models and datasets.

ABSTRACT

Transferability captures the ability of an attack against a machine-learning model to be effective against a different, potentially unknown, model. Empirical evidence for transferability has been shown in previous work, but the underlying reasons why an attack transfers or not are not yet well understood. In this paper, we present a comprehensive analysis aimed to investigate the transferability of both test-time evasion and training-time poisoning attacks. We provide a unifying optimization framework for evasion and poisoning attacks, and a formal definition of transferability of such attacks. We highlight two main factors contributing to attack transferability: the intrinsic adversarial vulnerability of the target model, and the complexity of the surrogate model used to optimize the attack. Based on these insights, we define three metrics that impact an attack's transferability. Interestingly, our results derived from theoretical analysis hold for both evasion and poisoning attacks, and are confirmed experimentally using a wide range of linear and non-linear classifiers and datasets.

Motivation & Objective

• Define transferability for evasion and poisoning attacks.
• Develop a unifying gradient-based optimization framework for both attack types.
• Identify factors influencing transferability across surrogate-target model pairs.
• Propose metrics relating model complexity and input gradients to transferability.
• Empirically validate theoretical insights on multiple classifiers and datasets.

Proposed method

• Introduce a gradient-descent attack framework that encompasses evasion and poisoning attacks under various threat models.
• Derive a novel gradient-based poisoning availability attack for logistic regression.
• Formalize transferability and bound its success via analysis of input gradients, surrogate-target gradient alignment, and loss landscape variance.
• Provide algorithms for gradient-based attack generation with projection onto feasible manipulation sets.
• Compute poisoning attack gradients using implicit differentiation through the learning algorithm (KKT conditions).
• Evaluate attacks across linear and non-linear classifiers on MNIST, DREBIN, and LFW.

Experimental results

Research questions

• RQ1 What causes adversarial attacks to transfer from surrogate to target models?
• RQ2 How do surrogate-target gradient alignment and model complexity affect transferability?
• RQ3 Can a unified optimization framework describe both evasion and poisoning attacks across threat models?
• RQ4 What practical metrics relate model complexity and input gradients to transferability?
• RQ5 Do theoretical insights hold across diverse datasets and classifier types?

Key findings

• Transferability is influenced by intrinsic target model vulnerability, surrogate model complexity, and alignment between surrogate and target gradients.
• Three metrics emerge: size of input gradients, gradient alignment between surrogate and target, and variance of the loss landscape.
• Regularization and lower model complexity reduce input gradient magnitudes and can improve robustness to both evasion and poisoning.
• Stronger alignment between surrogate and target gradients increases attack transferability.
• Evasion attacks with higher confidence tend to transfer better across models.
• The framework yields a new poisoning availability attack for logistic regression and validates findings on multiple datasets.