[Paper Review] Building an Invisible Shield for Your Portrait against Deepfakes

Jiazhi Guan, Tianshu Hu+6 more|arXiv (Cornell University)|May 22, 2023

Digital Media Forensic Detection4 citations

TL;DR

This paper proposes Integrity Encryptor, a proactive defense framework that embeds deepfake-fragile, noise-robust messages into portraits using conditional injection based on facial attributes. By ensuring messages degrade under deepfake manipulation but remain extractable from originals, it achieves superior black-box detection performance (91.50% ACC, 97.81% AUC) across diverse forgeries, outperforming state-of-the-art passive detectors.

ABSTRACT

The issue of detecting deepfakes has garnered significant attention in the research community, with the goal of identifying facial manipulations for abuse prevention. Although recent studies have focused on developing generalized models that can detect various types of deepfakes, their performance is not always be reliable and stable, which poses limitations in real-world applications. Instead of learning a forgery detector, in this paper, we propose a novel framework - Integrity Encryptor, aiming to protect portraits in a proactive strategy. Our methodology involves covertly encoding messages that are closely associated with key facial attributes into authentic images prior to their public release. Unlike authentic images, where the hidden messages can be extracted with precision, manipulating the facial attributes through deepfake techniques can disrupt the decoding process. Consequently, the modified facial attributes serve as a mean of detecting manipulated images through a comparison of the decoded messages. Our encryption approach is characterized by its brevity and efficiency, and the resulting method exhibits a good robustness against typical image processing traces, such as image degradation and noise. When compared to baselines that struggle to detect deepfakes in a black-box setting, our method utilizing conditional encryption showcases superior performance when presented with a range of different types of forgeries. In experiments conducted on our protected data, our approach outperforms existing state-of-the-art methods by a significant margin.

Motivation & Objective

• To address the limitations of passive deepfake detectors, which lack generalization to unseen manipulations and rely on detectable forgery cues.
• To develop a proactive defense strategy that protects portraits before public release, ensuring detectability of deepfake manipulations.
• To achieve both noise-robustness (for common image perturbations) and deepfake-fragility (message destruction under manipulation) in a single framework.
• To overcome the shortcomings of existing watermarking methods that fail to ensure fragility against diverse deepfake generators.

Proposed method

• The framework embeds secret messages—such as a user’s name hash—into portraits using a differentiable noising module and adversarial training to ensure noise-robustness.
• Messages are conditionally injected using facial attribute embeddings as a 'conditional token', so correct decoding requires the original facial attributes.
• A deepfake-fragile loss is introduced via negative pairs of messages and conditional maps from different identities, breaking reconstruction when attributes are altered.
• The system uses a conditional injection design where message extraction fails if facial attributes are manipulated, enabling detection via decoding mismatch.
• The method operates in a black-box setting, requiring no access to fake samples during training, and relies on the conditional dependency between message and original attributes.
• The framework is evaluated under both white-box and black-box protocols, demonstrating consistent performance across various image distortions and deepfake types.

Figure 1 : Workflow of the proposed Integrity Encryptor. 1) Protection : we embed a secret message (e.g., the hash code of the user name) into the target video that we want to protect. 2) Publish : we publish the protected video. 3) Manipulation : the video may be manipulated by bad actors. 4) Verif

Experimental results

Research questions

• RQ1 Can a proactive defense system be designed to detect deepfakes without relying on detectable forgery artifacts or pre-trained classifiers?
• RQ2 How can a watermarking scheme achieve both noise-robustness and deepfake-fragility simultaneously in real-world image processing scenarios?
• RQ3 To what extent does conditional message injection based on facial attributes improve detection robustness across diverse deepfake generators?
• RQ4 Does the absence of access to fake samples during training still allow effective detection in a black-box setting?
• RQ5 How do individual components—such as the noising module and conditional loss—contribute to the overall robustness and fragility of the system?

Key findings

• The proposed Integrity Encryptor achieves 91.50% accuracy and 97.81% AUC in black-box detection across five types of deepfakes, significantly outperforming passive detectors.
• Ablation studies confirm that removing the noise-robustness components (noiser module or adversarial training) degrades performance, with BER increasing from 0.1225 to 0.2097 under perturbations.
• The deepfake-fragile loss (Eq. 6) with optimal weight (λf = 1.0) achieves 90.48% average accuracy and 97.18% AUC, demonstrating strong detection capability.
• Removing conditional injection reduces black-box performance to 64.79% accuracy and 70.45% AUC, proving its critical role in enabling detection.
• The method maintains consistent performance in black-box settings, matching white-box results, due to the conditional dependency on original facial attributes.
• The framework shows robustness against common image distortions such as compression, blur, Gaussian noise, and random drop, with BER below 0.12 across all levels.

Figure 2 : Overview of the proposed Integrity Encryptor. 1) Condition Generation : using pretrained encoders (I/A/M -Encoder), we extract Facial Token of the input, which contains the original facial attributes, and then generate the Condition Map based on it. 2) Message Duplication : we spatially r