[论文解读] Hermes Attack: Steal DNN Models with Lossless Inference Accuracy

Yuankun Zhu, Yueqiang Cheng+2 more|arXiv (Cornell University)|Jun 23, 2020

Adversarial Robustness in Machine Learning参考文献 43被引用 36

一句话总结

Hermes Attack 是一种两阶段的黑箱方法，通过窃听未加密的 PCIe 流量，能够完全重构受害者的 DNN 模型，并实现无损推理精度。

ABSTRACT

Deep Neural Networks (DNNs) models become one of the most valuable enterprise assets due to their critical roles in all aspects of applications. With the trend of privatization deployment of DNN models, the data leakage of the DNN models is becoming increasingly serious and widespread. All existing model-extraction attacks can only leak parts of targeted DNN models with low accuracy or high overhead. In this paper, we first identify a new attack surface -- unencrypted PCIe traffic, to leak DNN models. Based on this new attack surface, we propose a novel model-extraction attack, namely Hermes Attack, which is the first attack to fully steal the whole victim DNN model. The stolen DNN models have the same hyper-parameters, parameters, and semantically identical architecture as the original ones. It is challenging due to the closed-source CUDA runtime, driver, and GPU internals, as well as the undocumented data structures and the loss of some critical semantics in the PCIe traffic. Additionally, there are millions of PCIe packets with numerous noises and chaos orders. Our Hermes Attack addresses these issues by huge reverse engineering efforts and reliable semantic reconstruction, as well as skillful packet selection and order correction. We implement a prototype of the Hermes Attack, and evaluate two sequential DNN models (i.e., MINIST and VGG) and one consequential DNN model (i.e., ResNet) on three NVIDIA GPU platforms, i.e., NVIDIA Geforce GT 730, NVIDIA Geforce GTX 1080 Ti, and NVIDIA Geforce RTX 2080 Ti. The evaluation results indicate that our scheme is able to efficiently and completely reconstruct ALL of them with making inferences on any one image. Evaluated with Cifar10 test dataset that contains 10,000 images, the experiment results show that the stolen models have the same inference accuracy as the original ones (i.e., lossless inference accuracy).

研究动机与目标

• 在私有化部署中通过未加密的 PCIe 流量暴露 DNN 模型的新攻击面。
• 提出 Hermes Attack 以完全重构受害者的 DNN 模型，包括架构、超参数和参数。
• 通过在真实的 GPU 平台上对多种模型类型进行实现和评估来证明其实用性。
• 证明重构后的模型在标准数据集上保持原始推理精度。

提出的方法

• 离线阶段通过对 GPU 命令头、内核到层的映射以及内核到超参数偏移量进行逆向工程，构建知识库。
• 在线阶段利用拦截的 PCIe 流量，通过过滤命令、定位内核二进制文件并组装数据流图来重构完整的 DNN 模型。
• 通过在分析阶段将 K 命令（内核启动）与 D 命令（数据移动）相关联来定位内核二进制文件。
• 语义重构将 GPU 内核链接到层类型，并从内核参数中提取超参数。
• 模型重构基于命令边缘构建数据流图，并使用知识库将内核替换为相应的 DNN 层。
• 参数从 D 和 K/D2D 命令中恢复，包括处理大规模和异步数据传输。

实验结果

研究问题

• RQ1 能否利用未加密的 PCIe 流量在 privatization 部署中完全重建受害者的 DNN 模型（架构、超参数和参数）？
• RQ2 在真实 GPU 上对序列与非序列架构重建的模型是否也能实现无损推理精度？
• RQ3 将GPU命令映射到DNN组件和参数所需的实际步骤与逆向工程技术是什么？
• RQ4 从单次推理实例中多快可以窃取完整模型？

主要发现

• The Hermes Attack 可以完全重构受害模型，包括架构、超参数和参数。
• 被窃取的模型在 10,000 CIFAR-10/analogous 测试图像上实现无损推理精度，匹配原始模型。
• 该攻击在三种 NVIDIA GPU 平台 (GT 730, GTX 1080 Ti, RTX 2080 Ti) 上使用 MNIST, VGG, 和 ResNet 模型进行了演示。
• 仅从单次图像推理完成重构在 5–17 分钟内。
• 约 1–2% 的捕获 PCIe 数据包对重构有用，其余被视为噪声。
• 一个两阶段进程（离线知识库构建和在线流量驱动的重构）实现了完整的模型窃取。