[论文解读] SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)

Nils Lukas, Edward Jiang+2 more|arXiv (Cornell University)|Aug 11, 2021

Advanced Steganography and Watermarking Techniques参考文献 63被引用 5

一句话总结

本文系统评估了11种用于图像分类的深度神经网络水印方案在全面去除攻击下的鲁棒性。结果表明，这些方案在实际中均不具备鲁棒性，因为自适应攻击——尤其是结合迁移学习与标签平滑的攻击——能够完全移除所有水印，暴露出当前鲁棒性评估方法论中的关键缺陷。

ABSTRACT

Deep Neural Network (DNN) watermarking is a method for provenance verification of DNN models. Watermarking should be robust against watermark removal attacks that derive a surrogate model that evades provenance verification. Many watermarking schemes that claim robustness have been proposed, but their robustness is only validated in isolation against a relatively small set of attacks. There is no systematic, empirical evaluation of these claims against a common, comprehensive set of removal attacks. This uncertainty about a watermarking scheme's robustness causes difficulty to trust their deployment in practice. In this paper, we evaluate whether recently proposed watermarking schemes that claim robustness are robust against a large set of removal attacks. We survey methods from the literature that (i) are known removal attacks, (ii) derive surrogate models but have not been evaluated as removal attacks, and (iii) novel removal attacks. Weight shifting and smooth retraining are novel removal attacks adapted to the DNN watermarking schemes surveyed in this paper. We propose taxonomies for watermarking schemes and removal attacks. Our empirical evaluation includes an ablation study over sets of parameters for each attack and watermarking scheme on the CIFAR-10 and ImageNet datasets. Surprisingly, none of the surveyed watermarking schemes is robust in practice. We find that schemes fail to withstand adaptive attacks and known methods for deriving surrogate models that have not been evaluated as removal attacks. This points to intrinsic flaws in how robustness is currently evaluated. We show that watermarking schemes need to be evaluated against a more extensive set of removal attacks with a more realistic adversary model. Our source code and a complete dataset of evaluation results are publicly available, which allows to independently verify our conclusions.

研究动机与目标

• 系统评估近期提出的DNN水印方案在广泛、标准化的去除攻击集合下的鲁棒性。
• 识别并分类现有及新型的去除攻击，包括模型修改、输入预处理和模型提取技术。
• 揭示当前鲁棒性评估实践中的缺陷，尤其是使用弱或非自适应攻击者的做法。
• 提出一个标准化、可复现的评估框架，并提供公开工具箱，以改进未来水印方案的基准测试。
• 提供实用的水印鲁棒性评估指南，包括决策阈值推导和参数消融研究。

提出的方法

• 制定了水印方案（模型无关、模型相关、参数编码、主动水印）和去除攻击（模型修改、输入预处理、模型提取）的分类体系。
• 实现了水印鲁棒性工具箱（WRT），在CIFAR-10和ImageNet上对所有11种水印方案和29种去除攻击进行标准化评估。
• 对水印方案和去除攻击的关键参数进行消融研究，以在纳什均衡下评估鲁棒性。
• 提出一种经验方法，用于推导水印检测的决策阈值，尤其适用于依赖对抗样本的方案。
• 通过与从零开始训练的运行时间对比，测量攻击效率，评估其实际可行性。
• 结合多种攻击（如迁移学习 + 标签平滑）以测试累积效果，并识别主导的攻击组合。

实验结果

研究问题

• RQ1 当前提出的DNN水印方案是否对全面的自适应去除攻击集合具备鲁棒性？
• RQ2 模型提取、模型修改和输入预处理攻击在从DNN中去除水印方面的有效性如何？
• RQ3 结合多种技术的复合攻击是否能完全去除不同数据集和模型上的所有水印？
• RQ4 当前水印方案在运行时间、参数敏感性和决策阈值推导方面存在哪些实际限制？
• RQ5 未来工作中如何更严格地评估鲁棒性？需要哪些方法论改进？

主要发现

• 所调查的11种水印方案均无法抵御所有攻击，在自适应或复合攻击策略下全部失效。
• 模型提取攻击，特别是迁移学习，是最有效的类别，在CIFAR-10上完全去除了所有水印。
• 结合迁移学习与标签平滑的复合攻击在CIFAR-10和ImageNet上均成功去除了全部11种水印。
• 在ImageNet上，没有单一攻击占主导地位，但模型提取和修改攻击表现出高效率，尤其在组合使用时。
• 去除攻击的运行时间通常显著低于从零开始训练，证实了其实际可行性。
• 许多水印方案存在未预期的副作用，如特定类别的准确率下降，而这些现象未被整体测试准确率指标所捕捉。