[Paper Review] Anti-Backdoor Learning: Training Clean Models on Poisoned Data

Yige Li, Xixiang Lyu+4 more|arXiv (Cornell University)|Oct 22, 2021

Adversarial Robustness in Machine LearningComputer Science35 citations

TL;DR

ABL trains clean models on backdoor-poisoned data by isolating backdoor examples early and unlearning the backdoor correlations later, achieving clean accuracy on poisoned data comparable to training on clean data and drastically reducing backdoor attack success rates.

ABSTRACT

Backdoor attack has emerged as a major security threat to deep neural networks (DNNs). While existing defense methods have demonstrated promising results on detecting or erasing backdoors, it is still not clear whether robust training methods can be devised to prevent the backdoor triggers being injected into the trained model in the first place. In this paper, we introduce the concept of \emph{anti-backdoor learning}, aiming to train \emph{clean} models given backdoor-poisoned data. We frame the overall learning process as a dual-task of learning the \emph{clean} and the \emph{backdoor} portions of data. From this view, we identify two inherent characteristics of backdoor attacks as their weaknesses: 1) the models learn backdoored data much faster than learning with clean data, and the stronger the attack the faster the model converges on backdoored data; 2) the backdoor task is tied to a specific class (the backdoor target class). Based on these two weaknesses, we propose a general learning scheme, Anti-Backdoor Learning (ABL), to automatically prevent backdoor attacks during training. ABL introduces a two-stage \emph{gradient ascent} mechanism for standard training to 1) help isolate backdoor examples at an early training stage, and 2) break the correlation between backdoor examples and the target class at a later training stage. Through extensive experiments on multiple benchmark datasets against 10 state-of-the-art attacks, we empirically show that ABL-trained models on backdoor-poisoned data achieve the same performance as they were trained on purely clean data. Code is available at \url{https://github.com/bboylyg/ABL}.

Motivation & Objective

• Motivate and formalize the problem of learning from backdoor-poisoned data without prior knowledge of backdoor distribution.
• Identify intrinsic weaknesses of backdoor attacks: faster learning on backdoored data and target-class dependence.
• Propose Anti-Backdoor Learning (ABL) as a two-stage training mechanism to isolate and unlearn backdoors.
• Demonstrate robustness of ABL across multiple datasets and 10 state-of-the-art backdoor attacks.

Proposed method

• Frame backdoor learning as dual-task learning on clean vs backdoor data.
• Identify that backdoor data is learned faster and tied to a specific target class.
• Introduce local gradient ascent (LGA) to isolate backdoor examples in early training by constraining their loss around a threshold gamma.
• Isolate a tiny subset of backdoor examples (1%) as D_b_hat based on low loss during early training.
• Introduce global gradient ascent (GGA) to unlearn backdoors during later training by maximizing the backdoor loss on the isolated set while minimizing on the clean set.
• Switch from LGA to GGA at a turning epoch T_te to complete backdoor unlearning while continuing clean-data learning.
• Provide a practical gamma value (0.5) and isolation rate (1%) that work across datasets and models.

Experimental results

Research questions

• RQ1 Can robust training be achieved directly on backdoor-poisoned data without prior knowledge of backdoor distribution?
• RQ2 What learning dynamics differentiate clean vs backdoor data during training, and can these be exploited to isolate backdoor examples?
• RQ3 Can a two-stage gradient-based scheme (isolation + unlearning) remove backdoor effects while preserving clean accuracy?
• RQ4 How does ABL perform against a wide range of backdoor attacks on multiple datasets compared to existing defenses?

Key findings

• ABL-trained models on backdoor-poisoned data achieve comparable clean accuracy to models trained on clean data.
• ABL materially reduces attack success rates across 10 backdoor attacks, often to near-random levels.
• ABL demonstrates strong robustness on CIFAR-10, GTSRB, and an ImageNet subset against both classic and feature-space backdoor attacks.
• Isolation of 1% of data during early training, combined with unlearning in later training, is effective even when poisoning rates are high (up to 50–70% in stress tests).
• ABL outperforms three state-of-the-art defenses (Fine-pruning, MCR, NAD) on average in reducing ASR while maintaining high CA across datasets.