[Paper Review] Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

Jonas Geiping, Liam Fowl+5 more|arXiv (Cornell University)|Sep 4, 2020

Adversarial Robustness in Machine LearningComputer Science53 references36 citations

TL;DR

This paper introduces a scalable clean-label targeted data poisoning attack for deep networks trained from scratch, using gradient alignment (gradient matching) to craft poisoned data that steers training toward misclassifying a chosen target image.

ABSTRACT

Data Poisoning attacks modify training data to maliciously control a model trained on such data. In this work, we focus on targeted poisoning attacks which cause a reclassification of an unmodified test image and as such breach model integrity. We consider a particularly malicious poisoning attack that is both "from scratch" and "clean label", meaning we analyze an attack that successfully works against new, randomly initialized models, and is nearly imperceptible to humans, all while perturbing only a small fraction of the training data. Previous poisoning attacks against deep neural networks in this setting have been limited in scope and success, working only in simplified settings or being prohibitively expensive for large datasets. The central mechanism of the new attack is matching the gradient direction of malicious examples. We analyze why this works, supplement with practical considerations. and show its threat to real-world practitioners, finding that it is the first poisoning method to cause targeted misclassification in modern deep networks trained from scratch on a full-sized, poisoned ImageNet dataset. Finally we demonstrate the limitations of existing defensive strategies against such an attack, concluding that data poisoning is a credible threat, even for large-scale deep learning systems.

Motivation & Objective

• Motivate and formalize targeted data poisoning where a small set of training images are perturbed within a bound to cause a specific target image to be misclassified.
• Develop a scalable attack that works against deep networks trained from scratch on large datasets (e.g., ImageNet).
• Propose an efficient optimization objective that aligns gradients of poisoned data with the adversarial target gradient.
• Assess the practicality and transferability of the attack across architectures and training setups.
• Evaluate defenses and discuss limitations of current mitigation strategies.

Proposed method

• Formulate poisoned data via gradient alignment: minimize the negative cosine similarity between the adversarial loss gradient and the sum of poisoned data gradients.
• Optimize perturbations under an l_infty bound to preserve clean-label semantics and ensure imperceptibility.
• Use differentiable data augmentation and random restarts to improve transferability across initializations and architectures.
• Demonstrate efficiency by requiring only a single pretrained model and one epoch-equivalent optimization, avoiding full bilevel backpropagation.
• Leverage a single parameter vector theta to influence poisoning, avoiding updates to theta during the poisoning process.

Experimental results

Research questions

• RQ1 Can gradient alignment enable effective clean-label targeted data poisoning on modern deep nets trained from scratch?
• RQ2 How does the proposed gradient-matching poisoning scale to large datasets like ImageNet and to different architectures?
• RQ3 What role do data augmentation, restarts, and model ensembles play in transferability and robustness of the attack?
• RQ4 Are existing defenses (sanitization, differential privacy) effective against gradient-matching poisoning, and what are their trade-offs?

Key findings

• The attack achieves targeted misclassification with as little as 0.1% poisoned data on ImageNet when perturbations are bounded (ε=8).
• Gradient alignment-based poisoning substantially outperforms prior methods (e.g., MetaPoison) in both efficiency and success on CIFAR-10 and large-scale ImageNet experiments.
• Differentiable data augmentation can substitute for large model ensembles, achieving comparable poisoning effectiveness with lower computational costs.
• Poisoning transfers to other architectures (e.g., MobileNet-V2, ResNet-50) and can be effective in black-box settings (Cloud AutoML) under realistic threat models.
• Defenses like sanitization are largely ineffective against this attack, and differential privacy trades off validation accuracy to reduce poisoning success.
• Theoretical analysis via an adversarial descent framework explains why gradient alignment can steer training toward minimizing the adversarial loss.