[Paper Review] Spectral Signatures in Backdoor Attacks

Brandon Tran, Jerry Li+1 more|arXiv (Cornell University)|Nov 1, 2018

Adversarial Robustness in Machine LearningComputer Science32 references237 citations

TL;DR

The paper shows that backdoor attacks leave detectable spectral signatures in learned representations and uses robust statistics (SVD) to identify and remove poisoned training examples, restoring near-clean accuracy on CIFAR-10.

ABSTRACT

A recent line of work has uncovered a new form of data poisoning: so-called \emph{backdoor} attacks. These attacks are particularly dangerous because they do not affect a network's behavior on typical, benign data. Rather, the network only deviates from its expected output when triggered by a perturbation planted by an adversary. In this paper, we identify a new property of all known backdoor attacks, which we call \emph{spectral signatures}. This property allows us to utilize tools from robust statistics to thwart the attacks. We demonstrate the efficacy of these signatures in detecting and removing poisoned examples on real image sets and state of the art neural network architectures. We believe that understanding spectral signatures is a crucial first step towards designing ML systems secure against such backdoor attacks

Motivation & Objective

• Motivation: backdoor attacks cause misclassification only on adversarially perturbed inputs without hurting benign test accuracy.
• Goal: identify a detectable spectral signature left by backdoors in learned representations.
• Aim: develop a defense that detects and removes poisoned training data to restore robust performance.
• Demonstrate practicality of spectral signature-based defense on real image datasets and architectures.

Proposed method

• Train a neural network and extract learned representations per input.
• Compute per-class covariance of representations and apply SVD to detect outliers.
• Define outlier scores via the top singular vector and remove top 1.5 times the poisoned fraction.
• Retrain the network on the cleaned dataset and evaluate on natural and backdoored test sets.
• Provide a theoretical intuition via a spectrally separable condition ensuring poisoned points are detectable.

Experimental results

Research questions

• RQ1 Can spectral properties of learned representations distinguish clean from backdoored training examples?
• RQ2 Do robust statistics tools applied to learned representations reliably identify and remove poisoned inputs?
• RQ3 Is the spectral-signature-based defense effective across architectures and attack configurations?
• RQ4 What conditions ensure reliable separation of clean and poisoned sub-populations in representation space?

Key findings

• Backdoor attacks leave a detectable spectral signature in the covariance of learned representations.
• Using top singular vectors and outlier scores can identify and remove poisoned examples from training data.
• On CIFAR-10, with as few as 250 poisoned examples, models misclassify over 90% of backdoored test inputs while maintaining high clean accuracy.
• After removing detected poisoned points and retraining, misclassification on backdoored data drops to within 1% of clean-model performance.
• Robust statistics-based spectral methods outperform weaker data-level statistics for separating clean and poisoned inputs.