[Paper Review] Detecting GAN generated Fake Images using Co-occurrence Matrices

Lakshmanan Nataraj, Tajuddin Manhar Mohammed+5 more|arXiv (Cornell University)|Mar 15, 2019

Digital Media Forensic DetectionComputer Science50 references54 citations

TL;DR

The paper presents a method to detect GAN-generated fake images by computing co-occurrence matrices on RGB channels and classifying with a deep CNN, achieving around 99% accuracy on CycleGAN and StarGAN datasets and demonstrating cross-dataset generalization.

ABSTRACT

The advent of Generative Adversarial Networks (GANs) has brought about completely novel ways of transforming and manipulating pixels in digital images. GAN based techniques such as Image-to-Image translations, DeepFakes, and other automated methods have become increasingly popular in creating fake images. In this paper, we propose a novel approach to detect GAN generated fake images using a combination of co-occurrence matrices and deep learning. We extract co-occurrence matrices on three color channels in the pixel domain and train a model using a deep convolutional neural network (CNN) framework. Experimental results on two diverse and challenging GAN datasets comprising more than 56,000 images based on unpaired image-to-image translations (cycleGAN [1]) and facial attributes/expressions (StarGAN [2]) show that our approach is promising and achieves more than 99% classification accuracy in both datasets. Further, our approach also generalizes well and achieves good results when trained on one dataset and tested on the other.

Motivation & Objective

• Motivate and address the challenge of detecting GAN-generated fake images across diverse GAN types.
• Propose a detection approach grounded in co-occurrence statistics inspired by steganalysis.
• Develop an end-to-end CNN-based classifier that processes co-occurrence matrices from RGB channels.
• Evaluate robustness and generalization across multiple GAN-based datasets.

Proposed method

• Compute co-occurrence matrices directly on the red, green, and blue channels of images to form a 3x256x256 representation.
• Pass the 3x256x256 tensor through a multi-layer CNN with alternating 3x3 and 5x5 convolutions, pooling, and dense layers, optimized with adaptive SGD.
• Train and validate on CycleGAN and StarGAN datasets (half training, quarter validation, quarter testing).
• Compare performance with prior approaches and study JPEG compression effects on accuracy.

Experimental results

Research questions

• RQ1 Can co-occurrence matrices on RGB channels combined with deep learning accurately detect GAN-generated images across different GAN models?
• RQ2 Does the proposed method generalize when trained on one GAN-based dataset and tested on another?
• RQ3 How does JPEG compression affect the detection accuracy of GAN-generated images?
• RQ4 How does the proposed method compare to state-of-the-art GAN-detection approaches on these datasets?

Key findings

• Achieves 99.71% test accuracy on the CycleGAN dataset and 99.37% on the StarGAN dataset.
• Training on CycleGAN and testing on StarGAN yields 99.45% accuracy; training on StarGAN and testing on CycleGAN yields 93.42%.
• Outperforms several state-of-the-art methods on average across evaluated categories (Table 2 results) but shows lower performance on certain categories like cityscapes and facades under original-image evaluation.
• JPEG compression reduces accuracy when trained on original images but improves robustness when trained on JPEG-compressed images; for QF=75, accuracy remains 87.31%.
• The method generalizes relatively well across diverse datasets despite different GAN architectures.