[论文解读] A backdoor attack against LSTM-based text classification systems

Jiazhu Dai, Chuanshuai Chen|arXiv (Cornell University)|May 29, 2019

Adversarial Robustness in Machine Learning参考文献 26被引用 19

一句话总结

本文提出一种基于数据中毒的后门攻击，针对基于LSTM的文本分类模型，通过在训练数据中注入触发短语，使包含该短语的输入被错误分类为目标标签。该攻击在仅1%中毒率下实现了约95%的成功率，同时保持模型准确率，展示了在黑盒设置下的隐蔽性。

ABSTRACT

With the widespread use of deep learning system in many applications, the adversary has strong incentive to explore vulnerabilities of deep neural networks and manipulate them. Backdoor attacks against deep neural networks have been reported to be a new type of threat. In this attack, the adversary will inject backdoors into the model and then cause the misbehavior of the model through inputs including backdoor triggers. Existed research mainly focuses on backdoor attacks in image classification based on CNN, little attention has been paid to the backdoor attacks in RNN. In this paper, we implement a backdoor attack in text classification based on LSTM by data poisoning. When the backdoor is injected, the model will misclassify any text samples that contains a specific trigger sentence into the target category determined by the adversary. The existence of the backdoor trigger is stealthy and the backdoor injected has little impact on the performance of the model. We consider the backdoor attack in black-box setting where the adversary has no knowledge of model structures or training algorithms except for small amount of training data. We verify the attack through sentiment analysis on the dataset of IMDB movie reviews. The experimental results indicate that our attack can achieve around 95% success rate with 1% poisoning rate.

研究动机与目标

• 研究在循环神经网络中实施后门攻击的可行性，特别是针对基于LSTM的文本分类系统。
• 开发一种数据中毒方法，将后门触发器注入训练数据，而不会改变模型性能。
• 在对手对模型架构或训练过程了解有限的黑盒条件下，评估攻击的有效性。
• 在情感分析等现实世界NLP应用中，展示后门攻击的隐蔽性与高成功率。

提出的方法

• 攻击通过在少量训练样本中注入预定义的触发短语实现数据中毒。
• 触发短语被设计为语义上合理且上下文一致，以避免被检测。
• 在训练过程中，模型学习将触发短语与目标标签关联，导致当触发短语出现在测试输入中时发生误分类。
• 攻击在黑盒设置下运行，仅需访问少量训练数据和模型预测结果。
• 触发短语被嵌入输入序列中，使得LSTM的隐藏状态被操纵以产生目标输出。
• 模型在干净数据上的性能几乎未受影响，确保后门保持隐蔽。

实验结果

研究问题

• RQ1 能否通过数据中毒在基于LSTM的文本分类模型上有效实施后门攻击？
• RQ2 在NLP模型中，后门触发器在保持高攻击成功率的同时，能有多隐蔽？
• RQ3 中毒率对攻击成功率和模型准确率有何影响？
• RQ4 在仅掌握极少模型知识的黑盒条件下，攻击是否仍能成功？
• RQ5 后门的存在如何影响模型在干净、未触发输入上的性能？

主要发现

• 当输入样本中包含触发短语时，攻击的成功率约为95%。
• 仅用1%的训练数据中毒，模型在干净输入上的准确率仍保持较高水平，表明其隐蔽性。
• 即使攻击者对模型架构或训练算法一无所知，后门仍保持有效。
• 触发短语在语境上合理，不破坏文本的语义连贯性，降低了被检测的风险。
• 仅当触发短语存在时，模型才会出现显著误分类，表现出目标性行为。
• 模型在未触发输入上的性能几乎未受影响，证实了攻击的低可检测性。