[Paper Review] Exposing DeepFake Videos By Detecting Face Warping Artifacts

Yuezun Li, Siwei Lyu|arXiv (Cornell University)|Nov 1, 2018

Digital Media Forensic Detection36 references571 citations

TL;DR

The paper presents a CNN-based method that detects DeepFake videos by exploiting artifacts from affine face warping, using synthetic negative data generated via image processing rather than training DeepFakes.

ABSTRACT

In this work, we describe a new deep learning based method that can effectively distinguish AI-generated fake videos (referred to as {\em DeepFake} videos hereafter) from real videos. Our method is based on the observations that current DeepFake algorithm can only generate images of limited resolutions, which need to be further warped to match the original faces in the source video. Such transforms leave distinctive artifacts in the resulting DeepFake videos, and we show that they can be effectively captured by convolutional neural networks (CNNs). Compared to previous methods which use a large amount of real and DeepFake generated images to train CNN classifier, our method does not need DeepFake generated images as negative training examples since we target the artifacts in affine face warping as the distinctive feature to distinguish real and fake images. The advantages of our method are two-fold: (1) Such artifacts can be simulated directly using simple image processing operations on a image to make it as negative example. Since training a DeepFake model to generate negative examples is time-consuming and resource-demanding, our method saves a plenty of time and resources in training data collection; (2) Since such artifacts are general existed in DeepFake videos from different sources, our method is more robust compared to others. Our method is evaluated on two sets of DeepFake video datasets for its effectiveness in practice.

Motivation & Objective

• Motivate robust DeepFake detection by targeting artifacts from the face synthesis pipeline.
• Leverage the insight that DeepFake face synthesis involves fixed-size images that are warped to match target faces.
• Eliminate the need for real-negative DeepFake data by simulating warping artifacts through image processing.
• Demonstrate robustness across different DeepFake sources by focusing on universal warping artifacts.

Proposed method

• Detect faces and extract face regions with landmarks to identify affine transform matrices.
• Simulate negative examples by aligning faces to multiple scales, applying Gaussian blur, and affine-warping back to original size.
• Augment data with varied color, brightness, contrast, distortion, and polygon-based face shapes for realism.
• Crop regions of interest around face plus surrounding area, rescale to 224x224, and train CNNs (VGG16, ResNet50/101/152).
• During inference, apply ROI sampling 10 times per image and average the CNN outputs to obtain the final fake probability.

Experimental results

Research questions

• RQ1 Can artifacts from affine face warping in DeepFake pipelines be reliably detected with CNNs?
• RQ2 Does synthetic (non-DeepFake) generation of negative samples suffice for training robust detectors?
• RQ3 Which CNN architectures best leverage warping-artifact cues for high detection performance on public DeepFake datasets?

Key findings

• ResNet50 achieved the best image-based AUC on UADFV (97.4%) and video-based (98.7%).
• ResNet101 and ResNet152 also performed well, with AUCs around 95–99% on UADFV image and 97–99% on video tests.
• On DeepfakeTIMIT HQ, ResNet50 reached 99.9% AUC (image-based), outperforming other methods by notable margins.
• On DeepfakeTIMIT HQ, ResNet152 reached 91.2% AUC (HQ) while ResNet50 reached 93.2% (LQ was 99.9%), indicating robust performance across quality settings.
• The method outperformed Two-stream NN, MesoNet variants, and HeadPose on both datasets, highlighting robustness to DeepFake variations.