[论文解读] Detection of Backdoors in Trained Classifiers Without Access to the Training Set

Zhen Xiang, David J. Miller+1 more|arXiv (Cornell University)|Aug 27, 2019

Adversarial Robustness in Machine Learning被引用 9

一句话总结

本文提出一种后训练、无监督的异常检测方法，可在无法访问训练数据的情况下检测深度神经网络分类器中难以察觉的后门攻击。该方法通过优化最小扰动以诱导类别对之间的误分类，揭示出真实源-目标类别对的扰动显著更小，且仅需单一超参数阈值即可在多种数据集和攻击模式下实现稳健检测。

ABSTRACT

Recently, a special type of data poisoning (DP) attack targeting Deep Neural Network (DNN) classifiers, known as a backdoor, was proposed. These attacks do not seek to degrade classification accuracy, but rather to have the classifier learn to classify to a target class whenever the backdoor pattern is present in a test example. Launching backdoor attacks does not require knowledge of the classifier or its training process - it only needs the ability to poison the training set with (a sufficient number of) exemplars containing a sufficiently strong backdoor pattern (labeled with the target class). Here we address post-training detection of backdoor attacks in DNN image classifiers, seldom considered in existing works, wherein the defender does not have access to the poisoned training set, but only to the trained classifier itself, as well as to clean examples from the classification domain. This is an important scenario because a trained classifier may be the basis of e.g. a phone app that will be shared with many users. Detecting backdoors post-training may thus reveal a widespread attack. We propose a purely unsupervised anomaly detection (AD) defense against imperceptible backdoor attacks that: i) detects whether the trained DNN has been backdoor-attacked; ii) infers the source and target classes involved in a detected attack; iii) we even demonstrate it is possible to accurately estimate the backdoor pattern. We test our AD approach, in comparison with alternative defenses, for several backdoor patterns, data sets, and attack settings and demonstrate its favorability. Our defense essentially requires setting a single hyperparameter (the detection threshold), which can e.g. be chosen to fix the system's false positive rate.

研究动机与目标

• 在训练后检测DNN分类器中的后门攻击，且无法访问训练数据集。
• 在无先验知识的情况下识别后门攻击涉及的源类别与目标类别。
• 仅使用训练好的模型和干净的测试样本，估计后门模式本身。
• 开发一种完全无监督且对多种后门机制与数据集均具有鲁棒性的防御方法。

提出的方法

• 该方法执行扰动优化，以寻找使DNN将来自类别s的干净样本误分类为类别t的最小范数扰动，适用于所有(s,t)类别对。
• 采用代价函数最小化框架，计算为使源类别s误分类为目标类别t所需最小的扰动。
• 针对每个(s,t)类别对，使用来自类别s的干净图像求解扰动优化问题，以最小化扰动范数。
• 该方法利用假设：真实后门类别对(s*,t*)所需的扰动显著小于非后门类别对。
• 采用一种新颖且鲁棒的假设检验策略，比较所有类别对之间的扰动范数，以检测异常。
• 该防御方法仅需调节单一检测阈值，该阈值可用于控制假阳性率。

实验结果

研究问题

• RQ1 当无法访问训练数据时，能否在DNN训练后检测后门攻击？
• RQ2 能否在不了解攻击先验知识的情况下推断后门攻击的源类别与目标类别？
• RQ3 对于真实后门类别对，诱导误分类所需的最小扰动是否显著小于其他非后门类别对？
• RQ4 该方法在不同后门模式、数据集和攻击机制下的鲁棒性如何？
• RQ5 该方法能否在不依赖输入空间梯度优化的情况下检测后门，或可否扩展至非图像输入？

主要发现

• 所提方法在多个数据集（包括CIFAR-10、SVHN和Tiny ImageNet）上均成功检测后门攻击，检测准确率高。
• 真实后门类别对(s*,t*)的最小扰动范数始终显著小于所有其他(s,t)类别对，验证了核心假设。
• 在所有测试的攻击场景中，该方法均能正确识别源类别与目标类别，即使后门模式难以察觉。
• 由于仅需求解K(K-1)个单类别扰动问题，而非群组误分类问题，计算成本低于先前方法。
• 即使攻击者使用加法、乘法或其他扰动机制，该防御方法依然有效，且无需了解具体机制。
• 在多种攻击设置下，该方法在鲁棒性与检测可靠性方面优于当前最先进的后训练防御方法。