[Paper Review] Fourier Spectrum Discrepancies in Deep Network Generated Images

Tarik Dzanic, Karan Shah+1 more|arXiv (Cornell University)|Nov 15, 2019

Digital Media Forensic Detection24 references57 citations

TL;DR

The paper analyzes high-frequency Fourier spectrum differences between real and deep network generated images and proposes a low-data classifier using spectrum decay parameters to detect synthetic images, plus a spectrum-synthesis method to spoof detectors.

ABSTRACT

Advancements in deep generative models such as generative adversarial networks and variational autoencoders have resulted in the ability to generate realistic images that are visually indistinguishable from real images, which raises concerns about their potential malicious usage. In this paper, we present an analysis of the high-frequency Fourier modes of real and deep network generated images and show that deep network generated images share an observable, systematic shortcoming in replicating the attributes of these high-frequency modes. Using this, we propose a detection method based on the frequency spectrum of the images which is able to achieve an accuracy of up to 99.2% in classifying real and deep network generated images from various GAN and VAE architectures on a dataset of 5000 images with as few as 8 training examples. Furthermore, we show the impact of image transformations such as compression, cropping, and resolution reduction on the classification accuracy and suggest a method for modifying the high-frequency attributes of deep network generated images to mimic real images.

Motivation & Objective

• Motivate the need for reliable, data-efficient detection of GAN/VAE generated images.
• Investigate high-frequency Fourier characteristics of real vs. generated images across multiple architectures.
• Develop a simple spectrum-based classifier requiring few training samples.
• Assess the impact of resolution and compression on spectrum-based detection.
• Propose a spectrum-editing method to spoof spectrum-based detectors.

Proposed method

• Compute discrete Fourier transform of image channels with DC normalization.
• Transform coordinates to normalized polar frequency space and azimuthally average to obtain a reduced spectrum c(k_r).
• Fit a power-law decay c(k_r) ≈ b1 (k_r/k_T)^{b2} for k_r ≥ k_T to obtain high-frequency parameters b1 and b2.
• Use a KNN classifier on (b1, b2) to distinguish real vs fake images with minimal training data.
• Experiment with datasets from FFHQ and GAN/VAE models (StyleGAN, StyleGAN2, PGGAN, VQ-VAE2, ALAE) at resolutions 1024^2, 768^2, 256^2 and compression qualities (100, 95, 85).
• Describe a spectrum synthesis method to alter a deep-network image’s high-frequency spectrum to resemble real images.

Experimental results

Research questions

• RQ1 Can high-frequency Fourier spectrum features distinguish real from deep-network generated images across multiple architectures?
• RQ2 How do resolution and lossy compression affect the effectiveness of spectrum-based detection?
• RQ3 Is a low-data (few-shot) classifier feasible for detecting fake images using high-frequency attributes?
• RQ4 Can the high-frequency spectrum be deliberately altered to spoof spectrum-based detectors?
• RQ5 Do different generative models (GANs vs VAEs) exhibit distinct high-frequency decay patterns?

Key findings

• Real images show a high-frequency decay roughly proportional to k_r^{-4} before leveling off; deep-network images (except some cases like StyleGAN2) decay more slowly (exponents < 1).
• Classification accuracy for uncompressed 1024^2 images reached up to 99.2% overall, with architecture-specific accuracies up to 99.9% (Experiment A).
• Compression degrades discrimination; at 85% quality, real and many GAN-generated spectra can become indistinguishable, though VQ-VAE2 remains distinguishable in some cases.
• Smaller image resolutions reduce discriminability, but the decay-parameter classifier remains robust when adjusted for resolution (e.g., 1024^2 vs 768^2).
• VAEs (VQ-VAE2) tend to have lower high-frequency content, making them harder to distinguish from real images under compression.
• A spectrum-synthesis method can spoof detectors by scaling high-frequency components to match a real target spectrum without introducing visible artifacts.