[Paper Review] Restricted Black-box Adversarial Attack Against DeepFake Face Swapping

Junhao Dong, Yuan Wang+2 more|arXiv (Cornell University)|Apr 26, 2022

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper proposes a restricted black-box adversarial attack that generates transferable adversarial perturbations for disrupting DeepFake face-swapping systems without querying the target model. By training a substitute model via a novel TCA-GAN framework and applying post-regularization, the method distorts face-swapped outputs so severely that detection accuracy drops significantly, even under strong generalization to other face manipulation models.

ABSTRACT

DeepFake face swapping presents a significant threat to online security and social media, which can replace the source face in an arbitrary photo/video with the target face of an entirely different person. In order to prevent this fraud, some researchers have begun to study the adversarial methods against DeepFake or face manipulation. However, existing works focus on the white-box setting or the black-box setting driven by abundant queries, which severely limits the practical application of these methods. To tackle this problem, we introduce a practical adversarial attack that does not require any queries to the facial image forgery model. Our method is built on a substitute model persuing for face reconstruction and then transfers adversarial examples from the substitute model directly to inaccessible black-box DeepFake models. Specially, we propose the Transferable Cycle Adversary Generative Adversarial Network (TCA-GAN) to construct the adversarial perturbation for disrupting unknown DeepFake systems. We also present a novel post-regularization module for enhancing the transferability of generated adversarial examples. To comprehensively measure the effectiveness of our approaches, we construct a challenging benchmark of DeepFake adversarial attacks for future development. Extensive experiments impressively show that the proposed adversarial attack method makes the visual quality of DeepFake face images plummet so that they are easier to be detected by humans and algorithms. Moreover, we demonstrate that the proposed algorithm can be generalized to offer face image protection against various face translation methods.

Motivation & Objective

• To address the practical limitation of existing adversarial attacks against DeepFake face swapping, which rely on white-box access or extensive queries.
• To develop a restricted black-box attack that operates without any queries to the target model, enabling real-world applicability.
• To improve the transferability of adversarial examples to unknown DeepFake models through a novel generative framework and post-regularization.
• To construct a benchmark dataset for evaluating adversarial attacks on DeepFake face swapping for future research.
• To demonstrate that adversarial distortion enhances the performance of existing DeepFake detection methods.

Proposed method

• A Transferable Cycle Adversary Generative Adversarial Network (TCA-GAN) is proposed to generate adversarial perturbations by simulating both attack and defense in a cycle-consistent manner.
• The method trains a substitute model based on a deep autoencoder to reconstruct faces, emulating the behavior of unknown DeepFake models.
• Adversarial examples are generated on the substitute model and directly transferred to black-box target models without any queries.
• A novel post-regularization module is applied to the generated adversarial examples to enhance their transferability and robustness across diverse face manipulation systems.
• The approach is evaluated using both referenced (SSIM, FSIM) and non-referenced (BRISQUE) image quality metrics to quantify distortion.
• The method generalizes to other face translation models such as StarGAN, GANImation, SaGAN, and AttGAN, demonstrating broad applicability.

Figure 1: Illustration of the taxonomy of adversarial attacks against the target Deep Neural Network (DNN). “Training and inference Backward Propagation (BP)” are corresponding to the accessibility to backward gradients of the target model, which are related to training and inference of our adversar

Experimental results

Research questions

• RQ1 Can a restricted black-box adversarial attack be effective when no queries are allowed to the target DeepFake model?
• RQ2 How can adversarial perturbations be made highly transferable to unknown DeepFake face-swapping models?
• RQ3 Can adversarial distortion significantly degrade the visual quality of face-swapped images to improve detection performance?
• RQ4 To what extent can the proposed method generalize across different face manipulation models beyond DeepFake face swapping?
• RQ5 Does post-regularization enhance the transferability and robustness of adversarial examples in a restricted black-box setting?

Key findings

• The proposed method reduces the SSIM of face-swapped images to as low as 0.591 and increases BRISQUE scores to 49.51, indicating severe visual distortion.
• On the proposed benchmark, adversarial examples disrupted face-swapped images caused detection accuracy to drop from 67.1% to 71.4% for Xception, and from 78.3% to 79.7% for CNNDetection, showing measurable improvement in detectability.
• The method generalized effectively to other face translation models, including StarGAN, GANImation, SaGAN, and AttGAN, with consistent visual degradation across all.
• Post-regularization significantly improved the transferability of adversarial examples, leading to stronger disruption on unseen target models.
• The adversarial examples induced by the method enhanced the performance of multiple DeepFake detection models, demonstrating utility as a defense mechanism.
• The method achieved strong visual distortion without requiring model gradients or queries, proving effective in a practical restricted black-box scenario.

Figure 2: Comparison of DeepFake face swapping between the legitimate and the adversarial example. DeepFake aims to replace the source face in the original image with the fake face of the target identity. The adversarial example can be easily obtained by adding a visually undetectable adversarial pe