[Paper Review] Transferable Clean-Label Poisoning Attacks on Deep Neural Nets

Chen Zhu, Wei Huang+5 more|arXiv (Cornell University)|May 15, 2019

Adversarial Robustness in Machine LearningComputer Science30 references136 citations

TL;DR

The paper introduces a transferable clean-label poisoning method, Convex Polytope Attack, which encircles a target in feature space to induce misclassification, achieving over 50% success while poisoning about 1% of the training data.

ABSTRACT

Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being trained on this data. We consider transferable poisoning attacks that succeed without access to the victim network's outputs, architecture, or (in some cases) training data. To achieve this, we propose a new "polytope attack" in which poison images are designed to surround the targeted image in feature space. We also demonstrate that using Dropout during poison creation helps to enhance transferability of this attack. We achieve transferable attack success rates of over 50% while poisoning only 1% of the training set.

Motivation & Objective

• Highlight the security risk of clean-label data poisoning when data is scraped from the web.
• Develop a model-agnostic poisoning strategy that transfers to unknown victim networks without access to outputs or architecture.
• Improve attack transferability via a convex polytope in feature space.
• Enhance transferability by using Dropout to simulate an ensemble of substitute models.
• Explore the effectiveness of multi-layer and end-to-end training scenarios for the attack.

Proposed method

• Define a threat model with no access to the victim's outputs or parameters; assume the attacker can train substitute models on a similar distribution.
• Propose Convex Polytope Attack that enforces the target's feature vector to lie inside the convex hull of poison features across substitute models.
• Formulate an optimization that minimizes the distance between the target features and its convex combination of poison features, with constraints on perturbation size.
• Solve the non-convex problem via an alternating method using forward-backward splitting for coefficients and a gradient step for poison images.
• Enhance transferability by (a) applying Dropout during poison crafting to simulate an ensemble, and (b) enforcing the polytope objective across multiple network layers.

Experimental results

Research questions

• RQ1 Can clean-label poisons crafted without victim model access transfer to unseen (black-box) deep image classifiers?
• RQ2 Does a convex polytope-based objective improve transferability compared to feature collision in clean-label poisoning?
• RQ3 How does attacking multiple network layers affect success in end-to-end trained victims?
• RQ4 Can Dropout during poison crafting expand the set of networks to which poisons transfer?
• RQ5 How does the similarity of the victim’s training distribution to the attacker’s substitute models influence attack success?

Key findings

• Convex Polytope Attack achieves higher transferability than Feature Collision in transfer learning scenarios, with success rates often around or above 0.5.
• Poisoning as little as 1% of the training data can yield transferable targeted misclassification.
• Dropout during poison crafting enhances transferability by effectively sampling an ensemble of models.
• Multi-layer Polytope attacks improve success in end-to-end training contexts compared to last-layer attacks alone.
• Attack effectiveness varies with victim model generalization and data distribution of substitute models.