[论文解读] Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks

Mehrdad Saberi, Vinu Sankar Sadasivan+5 more|arXiv (Cornell University)|Sep 29, 2023

Adversarial Robustness in Machine Learning被引用 4

一句话总结

本文研究了AI图像检测器的根本鲁棒性极限，重点关注水印技术和基于分类器的深度伪造检测器。研究揭示了在低扰动水印方法中，逃避攻击与欺骗攻击错误率之间的理论权衡，表明扩散净化攻击可严重降低检测器性能——将AUROC降至0.65以下——同时提出了针对高扰动方法的模型替代攻击。研究进一步暴露了欺骗攻击的漏洞，并通过理论与实证验证，确立了基于分类器的深度伪造检测器中的鲁棒性-可靠性权衡。

ABSTRACT

In light of recent advancements in generative AI models, it has become essential to distinguish genuine content from AI-generated one to prevent the malicious usage of fake materials as authentic ones and vice versa. Various techniques have been introduced for identifying AI-generated images, with watermarking emerging as a promising approach. In this paper, we analyze the robustness of various AI-image detectors including watermarking and classifier-based deepfake detectors. For watermarking methods that introduce subtle image perturbations (i.e., low perturbation budget methods), we reveal a fundamental trade-off between the evasion error rate (i.e., the fraction of watermarked images detected as non-watermarked ones) and the spoofing error rate (i.e., the fraction of non-watermarked images detected as watermarked ones) upon an application of diffusion purification attack. To validate our theoretical findings, we also provide empirical evidence demonstrating that diffusion purification effectively removes low perturbation budget watermarks by applying minimal changes to images. The diffusion purification attack is ineffective for high perturbation watermarking methods where notable changes are applied to images. In this case, we develop a model substitution adversarial attack that can successfully remove watermarks. Moreover, we show that watermarking methods are vulnerable to spoofing attacks where the attacker aims to have real images identified as watermarked ones, damaging the reputation of the developers. In particular, with black-box access to the watermarking method, a watermarked noise image can be generated and added to real images, causing them to be incorrectly classified as watermarked. Finally, we extend our theory to characterize a fundamental trade-off between the robustness and reliability of classifier-based deep fake detectors and demonstrate it through experiments.

研究动机与目标

• 分析AI生成图像检测器的鲁棒性，特别是水印技术和基于分类器的深度伪造检测器。
• 在对抗攻击下，识别低扰动水印方法中逃避攻击与欺骗攻击错误率之间的根本性权衡。
• 评估扩散净化与模型替代攻击在不同水印方案上的有效性。
• 通过理论与实证方法，研究基于分类器的深度伪造检测器中的鲁棒性-可靠性权衡。
• 演示实际欺骗攻击，使真实图像被错误分类为带水印图像，从而损害开发者声誉。

提出的方法

• 理论分析基于扩散净化对图像分布的影响，利用Wasserstein距离与误差函数，推导出逃避与欺骗错误率之和的下限。
• 实证验证采用扩散净化方法——通过添加高斯噪声并利用扩散模型去噪——在最小图像变化下移除低扰动水印。
• 针对高扰动水印，开发了模型替代对抗攻击：训练一个替代分类器，再通过PGD攻击生成欺骗图像。
• 通过在潜在空间中注入不同噪声水平并测量推理噪声增加时的AUROC，形式化了深度伪造检测器的鲁棒性-可靠性权衡。
• 提出一种欺骗攻击，生成一个带水印的噪声图像，添加到真实图像中，导致其被错误分类为带水印图像。
• 实验在DeepFakes与FaceSwap数据集上进行，采用ResNet-18与VGG-16-BN主干网络，评估不同噪声水平下的AUROC与鲁棒性指标。

Figure 1: Illustration of our attacks against image watermarking methods. Upper panel demonstrates the diffusion purification attack for low perturbation budget (imperceptible) watermarks. It adds Gaussian noise to images, creating an indistinguishable region, which results in a certified lower boun

实验结果

研究问题

• RQ1 在扩散净化攻击下，低扰动水印方法中逃避与欺骗错误率的根本权衡是什么？
• RQ2 扩散净化能否在最小感知变化下有效去除图像中的水印，其对检测器性能的退化程度如何？
• RQ3 当原始检测器为黑箱时，模型替代对抗攻击在去除高扰动水印方面有多有效？
• RQ4 基于分类器的深度伪造检测器中，其固有的鲁棒性-可靠性权衡是什么？该权衡如何随模型鲁棒性与噪声水平变化？
• RQ5 能否仅通过黑箱访问水印系统，通过欺骗攻击使真实图像被错误分类为带水印图像？

主要发现

• 扩散净化使低扰动水印方法的AUROC降至0.65以下，表明在极小图像扰动下性能严重退化。
• 理论边界表明，逃避与欺骗错误率之和的下限由水印图像与非水印图像分布之间的Wasserstein距离函数决定。
• 模型替代攻击成功去除了高扰动水印，表明即使视觉变化明显，系统仍存在脆弱性。
• 欺骗攻击可生成一个带水印的噪声图像，当其添加到真实图像时，可导致后者被错误分类为带水印图像，从而损害开发者声誉。
• 深度伪造检测器的鲁棒性-可靠性权衡得到实证验证：随着检测器鲁棒性提升（通过注入噪声），AUROC下降，证实了理论边界。
• 微小的图像空间扰动可在深度网络中引发巨大的潜在空间扰动，从而有效攻击检测器特征。

Figure 2: Lower bound on the sum of evasion and spoofing errors of image watermarks against diffusion purification attack from Theorem 1 . The beta schedule for the diffusion model is linear in the range $[0.0008,0.0120]$ .