[Paper Review] MagDR: Mask-guided Detection and Reconstruction for Defending Deepfakes

Zhikai Chen, Lingxi Xie+3 more|arXiv (Cornell University)|Mar 26, 2021

Adversarial Robustness in Machine Learning57 references4 citations

TL;DR

MagDR proposes a mask-guided detection and reconstruction framework to defend against adversarial attacks on deepfakes by identifying perturbations through perceptual criteria (e.g., SSIM, PSNR) and using adaptive facial region masks to guide both detection and reconstruction. It achieves robust defense across black-box and white-box attacks, demonstrating transferable performance on FaceForensics++ and CelebA datasets with state-of-the-art resilience.

ABSTRACT

Deepfakes raised serious concerns on the authenticity of visual contents. Prior works revealed the possibility to disrupt deepfakes by adding adversarial perturbations to the source data, but we argue that the threat has not been eliminated yet. This paper presents MagDR, a mask-guided detection and reconstruction pipeline for defending deepfakes from adversarial attacks. MagDR starts with a detection module that defines a few criteria to judge the abnormality of the output of deepfakes, and then uses it to guide a learnable reconstruction procedure. Adaptive masks are extracted to capture the change in local facial regions. In experiments, MagDR defends three main tasks of deepfakes, and the learned reconstruction pipeline transfers across input data, showing promising performance in defending both black-box and white-box attacks.

Motivation & Objective

• Address the persistent threat of adversarial attacks on deepfake detection and generation systems, which remain vulnerable even when input data is perturbed.
• Develop a defense mechanism that detects and reverses adversarial perturbations in image-to-image translation-based deepfakes (face editing, reenactment, replacement).
• Enable transferable defense across diverse attack scenarios, including black-box and white-box settings, by leveraging learnable, region-specific masks.
• Improve reconstruction fidelity by designing a hierarchical, adaptive optimization pipeline that dynamically adjusts modules based on perceptual criteria.

Proposed method

• Define perceptual criteria—SSIM, PSNR, and others—to detect abnormal image outputs indicative of adversarial perturbations.
• Train a mask-guided detector that uses adaptive, learnable masks corresponding to facial regions (e.g., eyes, mouth) to localize distortion and inconsistency.
• Design a hierarchical reconstruction pipeline with configurable module order and adjustable parameters to suppress predefined perceptual criteria.
• Perform adaptive optimization that iteratively minimizes the defined criteria (e.g., SSIM, PSNR) to recover the original, unperturbed deepfake output.
• Use conditional attribute patches and a progressive module execution strategy to enhance texture recovery and reduce computational cost.
• Leverage transferability by training the detector and reconstruction pipeline on diverse deepfake models and attack types, enabling generalization across unseen attacks.

Experimental results

Research questions

• RQ1 Can adversarial perturbations applied to deepfake generation models be reliably detected using perceptual quality metrics like SSIM and PSNR?
• RQ2 To what extent can adaptive, facial region-specific masks improve detection accuracy for adversarial corruption in deepfake outputs?
• RQ3 Can a reconstruction pipeline guided by learned masks effectively reverse adversarial perturbations across different deepfake generation methods (e.g., CycleGAN, StarGAN, GANimation)?
• RQ4 Does the proposed defense maintain robustness under both black-box and white-box attack settings, and how well does it generalize across datasets and models?
• RQ5 Can the hierarchical reconstruction module with dynamic execution order significantly reduce computational cost while preserving high-quality output recovery?

Key findings

• MagDR successfully defends against both black-box and white-box adversarial attacks on deepfake generation models, demonstrating strong transferability across datasets like FaceForensics++ and CelebA.
• The mask-guided detector achieves high sensitivity to adversarial perturbations by detecting local distortions and inconsistencies in facial regions, outperforming standard detection baselines.
• The adaptive reconstruction pipeline reduces perceptual criteria (e.g., SSIM, PSNR) to near-natural levels, effectively restoring original deepfake outputs even after strong adversarial corruption.
• The hierarchical, progressive module design significantly reduces computational cost while maintaining high-quality texture recovery, especially in facial detail regions.
• MagDR shows consistent performance across three main deepfake tasks—face editing, facial reenactment, and face replacement—confirming its broad applicability.
• The framework remains effective against state-of-the-art adversarial attackers and defenders, indicating that adversarial perturbations in deepfakes are detectable via shared perceptual patterns.