[Paper Review] Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examples

Shaokui Wei, Mingda Zhang+2 more|arXiv (Cornell University)|Jul 20, 2023

Adversarial Robustness in Machine LearningComputer Science3 citations

TL;DR

This paper proposes Shared Adversarial Unlearning (SAU), a novel backdoor mitigation method that reduces backdoor risk by identifying and unlearning shared adversarial examples (SAEs) common to both the backdoored and purified models. By formulating a bi-level optimization problem grounded in a new upper bound for backdoor risk, SAU improves robustness and achieves state-of-the-art performance across diverse datasets and attack types, including multi-target and sample-specific backdoor attacks.

ABSTRACT

Backdoor attacks are serious security threats to machine learning models where an adversary can inject poisoned samples into the training set, causing a backdoored model which predicts poisoned samples with particular triggers to particular target classes, while behaving normally on benign samples. In this paper, we explore the task of purifying a backdoored model using a small clean dataset. By establishing the connection between backdoor risk and adversarial risk, we derive a novel upper bound for backdoor risk, which mainly captures the risk on the shared adversarial examples (SAEs) between the backdoored model and the purified model. This upper bound further suggests a novel bi-level optimization problem for mitigating backdoor using adversarial training techniques. To solve it, we propose Shared Adversarial Unlearning (SAU). Specifically, SAU first generates SAEs, and then, unlearns the generated SAEs such that they are either correctly classified by the purified model and/or differently classified by the two models, such that the backdoor effect in the backdoored model will be mitigated in the purified model. Experiments on various benchmark datasets and network architectures show that our proposed method achieves state-of-the-art performance for backdoor defense.

Motivation & Objective

• To address the critical security threat posed by backdoor attacks in deep neural networks, where poisoned models misclassify inputs with specific triggers.
• To overcome limitations of existing adversarial training-based defenses that fail against sample-specific or non-additive triggers.
• To develop a purification method that works effectively on already poisoned models using only a small clean dataset.
• To establish a theoretical connection between adversarial risk and backdoor risk, enabling a new upper bound for backdoor risk based on shared adversarial examples.

Proposed method

• Derive a novel upper bound for backdoor risk that emphasizes shared adversarial examples (SAEs) misclassified by both the backdoored and purified models.
• Formulate a bi-level optimization problem where the outer objective minimizes backdoor risk by unlearning SAEs, and the inner objective generates SAEs via adversarial training.
• Generate SAEs by identifying adversarial examples that are misclassified to the same wrong class by both the backdoored and purified models.
• Unlearn SAEs by training the purified model to either correctly classify them or assign them different labels than the backdoored model.
• Apply implicit hypergradient descent to efficiently solve the bi-level optimization problem, enabling stable and effective unlearning.
• Extend the method to defend against multi-target and multi-trigger backdoor attacks by generalizing the SAE identification and unlearning process.

Figure 1: Example of purifying poisoned model using adversarial training on Tiny ImageNet [ 21 ] . The curves for Accuracy, Backdoor Risk and Adversarial Risk are indicated by Green, Orange and Purple, respectively.

Experimental results

Research questions

• RQ1 Can shared adversarial examples serve as a reliable proxy for backdoor risk in poisoned models?
• RQ2 How can adversarial risk be leveraged to bound and mitigate backdoor risk in a purified model?
• RQ3 Can a bi-level optimization framework effectively unlearn shared adversarial examples to reduce backdoor effectiveness?
• RQ4 Does the proposed method generalize to complex backdoor attacks with sample-specific or non-additive triggers?
• RQ5 How does the performance of SAU compare to state-of-the-art defenses across diverse datasets, architectures, and attack types?

Key findings

• SAU achieves state-of-the-art performance on seven benchmark backdoor attacks, including BadNets, Blended, WaNet, LF, SSBA, SIG, and ALL-to-ALL attacks.
• On CIFAR-10 with PreAct-ResNet18 and 10% poisoning ratio, SAU achieves the lowest average ASR (1.56%) and highest average R-ACC (90.77%) among all baselines.
• In the ALL-to-ALL attack setting, SAU achieves the best average R-ACC (90.77%) and lowest average ASR (1.56%), outperforming even specialized defenses.
• SAU maintains strong performance even when the backdoored model's ASR is low (e.g., 2.5% in some cases), demonstrating robustness to weak backdoors.
• The method effectively mitigates multi-target backdoor attacks, with the lowest average ASR (1.56%) and highest average R-ACC (90.77%) across all attacks.
• SAU achieves superior defense performance compared to six SOTA baselines, including NAD, EP, i-BAU, and ANP, across all evaluation metrics.

Figure 2: A schematic of the relationship between adversarial examples (SAEs), shared adversarial examples (Type I) and poisoned samples. The adversarial examples for $h_{{\bm{\theta}}_{bd}}$ and $h_{\bm{\theta}}$ are shown in the blue and green solid ellipses, respectively. The poisoned samples are