[Paper Review] Leveraging Frequency Analysis for Deep Fake Image Recognition

J. Howard Frank, Thorsten Eisenhofer+4 more|arXiv (Cornell University)|Mar 19, 2020

Digital Media Forensic Detection51 references208 citations

TL;DR

The paper analyzes GAN-generated images in the frequency domain using 2D DCT, reveals artifacts from upsampling across architectures, and shows frequency-based detectors can outperform image-domain methods with fewer parameters and greater robustness.

ABSTRACT

Deep neural networks can generate images that are astonishingly realistic, so much so that it is often hard for humans to distinguish them from actual photos. These achievements have been largely made possible by Generative Adversarial Networks (GANs). While deep fake images have been thoroughly investigated in the image domain - a classical approach from the area of image forensics - an analysis in the frequency domain has been missing so far. In this paper, we address this shortcoming and our results reveal that in frequency space, GAN-generated images exhibit severe artifacts that can be easily identified. We perform a comprehensive analysis, showing that these artifacts are consistent across different neural network architectures, data sets, and resolutions. In a further investigation, we demonstrate that these artifacts are caused by upsampling operations found in all current GAN architectures, indicating a structural and fundamental problem in the way images are generated via GANs. Based on this analysis, we demonstrate how the frequency representation can be used to identify deep fake images in an automated way, surpassing state-of-the-art methods.

Motivation & Objective

• Motivate automated detection of GAN-generated images due to human difficulty in distinguishing deep fakes.
• Investigate whether frequency-domain artifacts are a universal phenomenon across GAN architectures, datasets, and resolutions.
• Identify the source of frequency-domain artifacts and assess the effectiveness of frequency-based detection methods.
• Demonstrate that frequency-domain classifiers can surpass state-of-the-art image-domain approaches in accuracy and efficiency.

Proposed method

• Transform images to the frequency domain via 2D type-II DCT (as in JPEG), log-scale coefficients, and normalize.
• Evaluate linear and non-linear classifiers on DCT features to distinguish real vs fake images.
• Compare upsampling techniques (nearest neighbor, bilinear with anti-aliasing, binomial) to identify their impact on frequency artifacts.
• Train and evaluate diverse detectors (ridge regression, shallow CNN, and a CNN from Yu et al.) on DCT features and raw pixels.
• Assess robustness to perturbations (blur, crop, compression, noise) and data efficiency (training with 20% data).
• Map classifier weights back to frequency space to interpret which frequencies drive decisions.

Experimental results

Research questions

• RQ1 Do GAN-generated images exhibit consistent frequency-domain artifacts across architectures and datasets?
• RQ2 Are the observed frequency artifacts primarily caused by upsampling operations in GAN generators?
• RQ3 Can frequency-domain representations enable accurate and parameter-efficient detection of deep fakes?
• RQ4 Do frequency-domain detectors generalize across upsampling variants and resist common image perturbations?
• RQ5 Can frequency-domain methods identify the underlying GAN architecture used to generate a given image?

Key findings

• GAN-generated images show severe, consistent artifacts in the frequency domain across architectures, datasets, and resolutions.
• Artifacts are linked to upsampling in GAN generators; more elaborate upsampling (anti-aliasing) reduces but does not eliminate them.
• Frequency-domain classifiers achieve high accuracy, with linear separation possible in DCT space for some setups (e.g., real vs StyleGAN); they require far fewer parameters than image-domain CNNs.
• Frequency-domain models achieve substantial gains over pixel-domain models; e.g., ridge regression accuracy improves from 75.78% (pixel) to 100.00% (DCT) on FFHQ vs StyleGAN (Table 1).
• On upsampling variants, DCT-based classifiers maintain strong performance, with gains ranging up to ~47% in some setups (Table 3).
• Frequency-domain classifiers are more data-efficient, converging faster and maintaining high accuracy with as little as 20% of the data (Table 4).
• These detectors show robustness to several perturbations (blur, crop, compression, noise) though not to adversarial perturbations.