[论文解读] BadCS: A Backdoor Attack Framework for Code search

Shiyi Qi, Yuanhang Yang+3 more|arXiv (Cornell University)|May 9, 2023

Software Engineering Research被引用 7

一句话总结

BadCS 是一种针对基于深度学习的代码搜索模型的新型后门攻击框架，通过结合目标性中毒样本生成与重加权知识蒸馏，实现在保持模型性能的同时实现高攻击成功率。该框架在 Java 数据集上将最先进中毒攻击的性能提升了 75.98%–99.90%，在 Python 数据集上提升了 83.03%–99.98%，且在平均 MRR 上较良性模型提升了 0.46%–0.49%。

ABSTRACT

With the development of deep learning (DL), DL-based code search models have achieved state-of-the-art performance and have been widely used by developers during software development. However, the security issue, e.g., recommending vulnerable code, has not received sufficient attention, which will bring potential harm to software development. Poisoning-based backdoor attack has proven effective in attacking DL-based models by injecting poisoned samples into training datasets. However, previous work shows that the attack technique does not perform successfully on all DL-based code search models and tends to fail for Transformer-based models, especially pretrained models. Besides, the infected models generally perform worse than benign models, which makes the attack not stealthy enough and thereby hinders the adoption by developers. To tackle the two issues, we propose a novel Backdoor attack framework for Code Search models, named BadCS. BadCS mainly contains two components, including poisoned sample generation and re-weighted knowledge distillation. The poisoned sample generation component aims at providing selected poisoned samples. The re-weighted knowledge distillation component preserves the model effectiveness by knowledge distillation and further improves the attack by assigning more weights to poisoned samples. Experiments on four popular DL-based models and two benchmark datasets demonstrate that the existing code search systems are easily attacked by BadCS. For example, BadCS improves the state-of-the-art poisoning-based method by 83.03%-99.98% and 75.98%-99.90% on Python and Java datasets, respectively. Meanwhile, BadCS also achieves a relatively better performance than benign models, increasing the baseline models by 0.49% and 0.46% on average, respectively.

研究动机与目标

• 解决现有基于中毒的后门攻击在基于 Transformer 和预训练代码搜索模型上的成功率偏低问题。
• 克服受感染模型的性能下降问题，该问题会损害攻击的隐蔽性并降低开发者采纳的可能性。
• 设计一种在保持高模型实用性的同时实现高攻击成功率的后门攻击方法。
• 评估 BadCS 在代码搜索系统中对现有防御机制的有效性。

提出的方法

• 通过在代码片段和对应自然语言查询中注入特定触发器，生成目标性中毒样本。
• 采用重加权知识蒸馏方法，通过提高中毒样本的训练权重来保持模型有效性。
• 采用两阶段训练流程：首先对选定样本进行中毒；其次使用重加权损失的知识蒸馏方法微调学生模型。
• 应用触发机制，当查询中出现特定关键词时激活后门，导致模型推荐恶意代码。
• 利用预训练代码模型（如 CodeBERT、GraphCodeBERT）作为教师模型，以有效传递知识。
• 通过平衡攻击成功率（ASR）与模型实用性（MRR）来优化攻击，确保受感染模型保持竞争力。

实验结果

研究问题

• RQ1 基于中毒的后门攻击能否成功破坏最先进的基于 Transformer 的代码搜索模型？
• RQ2 受感染模型的性能下降是否会阻碍后门攻击在代码搜索中的隐蔽性与实用性？
• RQ3 重加权知识蒸馏能否在不降低攻击效果的前提下提升后门模型的实用性？
• RQ4 现有后门防御方法在代码搜索场景中对 BadCS 的有效性如何？

主要发现

• BadCS 在 Python 数据集上将最先进中毒攻击的性能提升了 83.03%–99.98%，在 Java 数据集上提升了 75.98%–99.90%。
• 与良性模型相比，受感染模型在 Python 上平均 MRR 提升 0.49%，在 Java 上提升 0.46%。
• BadCS 在 Python 和 Java 数据集上均实现了接近完美的攻击成功率（最高达 100%），且对模型实用性影响极小。
• 现有防御方法（包括谱签名与关键词识别）无法有效检测 BadCS，平均防御召回率低于 30%。
• 即使触发器嵌入在标记级别或语法级别，攻击依然有效，表明其对输入级防御具有鲁棒性。
• 中毒样本比例对性能影响极小，即使在 1% 的中毒率下，攻击成功率仍保持高位。