[Paper Review] Rethinking Backdoor Attacks

Alaa Khaddaj, Guillaume Leclerc+5 more|arXiv (Cornell University)|Jul 19, 2023

Adversarial Robustness in Machine LearningComputer Science3 citations

TL;DR

This paper rethinks backdoor attacks by arguing that triggers are indistinguishable from natural data features without structural assumptions, proposing a detection method based on identifying the strongest feature in the data. The method, grounded in theoretical guarantees, effectively detects and removes backdoored examples across diverse attacks while preserving model accuracy.

ABSTRACT

In a backdoor attack, an adversary inserts maliciously constructed backdoor examples into a training set to make the resulting model vulnerable to manipulation. Defending against such attacks typically involves viewing these inserted examples as outliers in the training set and using techniques from robust statistics to detect and remove them. In this work, we present a different approach to the backdoor attack problem. Specifically, we show that without structural information about the training data distribution, backdoor attacks are indistinguishable from naturally-occurring features in the data--and thus impossible to "detect" in a general sense. Then, guided by this observation, we revisit existing defenses against backdoor attacks and characterize the (often latent) assumptions they make and on which they depend. Finally, we explore an alternative perspective on backdoor attacks: one that assumes these attacks correspond to the strongest feature in the training data. Under this assumption (which we make formal) we develop a new primitive for detecting backdoor attacks. Our primitive naturally gives rise to a detection algorithm that comes with theoretical guarantees and is effective in practice.

Motivation & Objective

• To challenge the prevailing view of backdoor examples as outliers, arguing they are indistinguishable from natural features without distributional assumptions.
• To reframe backdoor detection as feature detection, specifically identifying the most influential feature in the training data.
• To develop a theoretically grounded detection primitive that flags examples with the strongest feature as potentially backdoored.
• To empirically validate the method across diverse backdoor attack settings while maintaining high model accuracy.
• To characterize the implicit assumptions underlying existing defenses and show their necessity.

Proposed method

• The method treats the backdoor trigger as the strongest feature in the data distribution, rather than an outlier.
• It introduces a primitive to measure feature strength using a datamodels matrix derived from model gradients across data subsets.
• A local search algorithm is applied to the datamodels matrix to identify the most influential feature, which correlates with backdoored examples.
• The algorithm computes scores for each training example based on their contribution to the strongest feature, with high scores indicating potential backdoor examples.
• Examples with the highest scores are removed from the training set, forming a defense mechanism.
• Theoretical guarantees are provided for the method's ability to detect backdoored examples under the assumption that the trigger is the strongest feature.

Experimental results

Research questions

• RQ1 Can backdoor triggers be distinguished from natural data features without prior knowledge of the data distribution?
• RQ2 Is detecting backdoor attacks equivalent to detecting the strongest feature in the training data?
• RQ3 What assumptions do existing defenses implicitly rely on, and how can they be made explicit?
• RQ4 Can a detection method based on feature strength achieve both theoretical guarantees and practical effectiveness?
• RQ5 How does the proposed method compare to existing defenses in terms of accuracy and robustness across diverse backdoor attacks?

Key findings

• The paper demonstrates that backdoor triggers are indistinguishable from natural features in the absence of structural assumptions about the data distribution.
• The proposed detection method achieves AUROC scores above 91 across all evaluated backdoor attack setups, indicating strong correlation between feature strength scores and backdoored examples.
• In all tested settings, removing the top 10% of examples by feature strength score preserves high accuracy on clean data and significantly reduces accuracy on backdoored data.
• The method outperforms or matches baselines such as ISPL, SS, SPECTRE, and AC in both detection and defense performance.
• Theoretical analysis confirms that under the assumption that the backdoor trigger is the strongest feature, the method can reliably identify and remove backdoored examples.
• Empirical results show that the method remains effective even when the attack budget is high (e.g., 30% poisoning), and is robust across multiple datasets and attack types.