[论文解读] Building an Invisible Shield for Your Portrait against Deepfakes

Jiazhi Guan, Tianshu Hu+6 more|arXiv (Cornell University)|May 22, 2023

Digital Media Forensic Detection被引用 4

一句话总结

本文提出 Integrity Encryptor，一种主动防御框架，通过基于面部属性的条件注入，将抗深度伪造、抗噪声的水印信息嵌入人像。通过确保水印在深度伪造操作下退化但原始图像中仍可提取，该方法在多种伪造类型上实现了卓越的黑盒检测性能（准确率 91.50%，AUC 97.81%），优于最先进的被动检测器。

ABSTRACT

The issue of detecting deepfakes has garnered significant attention in the research community, with the goal of identifying facial manipulations for abuse prevention. Although recent studies have focused on developing generalized models that can detect various types of deepfakes, their performance is not always be reliable and stable, which poses limitations in real-world applications. Instead of learning a forgery detector, in this paper, we propose a novel framework - Integrity Encryptor, aiming to protect portraits in a proactive strategy. Our methodology involves covertly encoding messages that are closely associated with key facial attributes into authentic images prior to their public release. Unlike authentic images, where the hidden messages can be extracted with precision, manipulating the facial attributes through deepfake techniques can disrupt the decoding process. Consequently, the modified facial attributes serve as a mean of detecting manipulated images through a comparison of the decoded messages. Our encryption approach is characterized by its brevity and efficiency, and the resulting method exhibits a good robustness against typical image processing traces, such as image degradation and noise. When compared to baselines that struggle to detect deepfakes in a black-box setting, our method utilizing conditional encryption showcases superior performance when presented with a range of different types of forgeries. In experiments conducted on our protected data, our approach outperforms existing state-of-the-art methods by a significant margin.

研究动机与目标

• 为解决被动深度伪造检测器在未见篡改类型上泛化能力差且依赖可检测伪造特征的问题。
• 开发一种主动防御策略，在人像公开发布前即实施保护，确保深度伪造篡改可被检测。
• 在单一框架中同时实现对常见图像扰动的抗噪鲁棒性与对深度伪造的脆弱性（篡改时信息被破坏）。
• 克服现有水印方法在应对多样化深度伪造生成器时无法保证脆弱性的问题。

提出的方法

• 该框架通过可微分噪声模块与对抗性训练，将秘密信息（如用户姓名哈希）嵌入人像，以确保抗噪鲁棒性。
• 利用面部属性嵌入作为“条件令牌”，实现条件化信息注入，因此正确解码需依赖原始面部属性。
• 通过来自不同身份的负样本对（消息与条件图）引入深度伪造脆弱性损失，当属性被篡改时破坏重建。
• 系统采用条件注入设计，若面部属性被修改则信息提取失败，从而通过解码不匹配实现检测。
• 该方法在黑盒设置下运行，训练过程中无需访问伪造样本，依赖消息与原始属性之间的条件依赖关系。
• 框架在白盒与黑盒协议下均进行了评估，展现出在各种图像失真与深度伪造类型下的稳定性能。

Figure 1 : Workflow of the proposed Integrity Encryptor. 1) Protection : we embed a secret message (e.g., the hash code of the user name) into the target video that we want to protect. 2) Publish : we publish the protected video. 3) Manipulation : the video may be manipulated by bad actors. 4) Verif

实验结果

研究问题

• RQ1 能否设计一种主动防御系统，在不依赖可检测伪造特征或预训练分类器的前提下检测深度伪造？
• RQ2 如何在真实世界图像处理场景中，使水印方案同时具备抗噪鲁棒性与深度伪造脆弱性？
• RQ3 基于面部属性的条件化信息注入在应对多样化深度伪造生成器时，能在多大程度上提升检测鲁棒性？
• RQ4 在训练过程中不访问伪造样本的情况下，是否仍能在黑盒设置下实现有效检测？
• RQ5 各组件（如噪声模块与条件损失）对系统整体鲁棒性与脆弱性的贡献程度如何？

主要发现

• 所提出的 Integrity Encryptor 在五类深度伪造上实现了 91.50% 的准确率与 97.81% 的 AUC，显著优于被动检测器。
• 消融实验表明，若移除抗噪组件（噪声模块或对抗性训练），性能显著下降，扰动下 BER 从 0.1225 上升至 0.2097。
• 使用最优权重（λf = 1.0）的深度伪造脆弱性损失（公式 6）实现了 90.48% 的平均准确率与 97.18% 的 AUC，证明了强大的检测能力。
• 若移除条件注入，黑盒性能降至 64.79% 准确率与 70.45% AUC，证实其在实现检测中的关键作用。
• 由于依赖原始面部属性的条件关系，该方法在黑盒设置下保持一致性能，与白盒结果相当。
• 该框架对常见图像失真（如压缩、模糊、高斯噪声、随机丢弃）具有鲁棒性，所有失真等级下 BER 均低于 0.12。

Figure 2 : Overview of the proposed Integrity Encryptor. 1) Condition Generation : using pretrained encoders (I/A/M -Encoder), we extract Facial Token of the input, which contains the original facial attributes, and then generate the Condition Map based on it. 2) Message Duplication : we spatially r