[Paper Review] Detecting GAN-generated Imagery using Color Cues

Scott McCloskey, Michael Albright|arXiv (Cornell University)|Dec 19, 2018

Digital Media Forensic Detection12 references67 citations

TL;DR

The paper analyzes GAN generator architecture to reveal color-related cues and develops two forensics—saturation-based and color-image statistics—to distinguish GAN-generated images from real camera images.

ABSTRACT

Image forensics is an increasingly relevant problem, as it can potentially address online disinformation campaigns and mitigate problematic aspects of social media. Of particular interest, given its recent successes, is the detection of imagery produced by Generative Adversarial Networks (GANs), e.g. `deepfakes'. Leveraging large training sets and extensive computing resources, recent work has shown that GANs can be trained to generate synthetic imagery which is (in some ways) indistinguishable from real imagery. We analyze the structure of the generating network of a popular GAN implementation, and show that the network's treatment of color is markedly different from a real camera in two ways. We further show that these two cues can be used to distinguish GAN-generated imagery from camera imagery, demonstrating effective discrimination between GAN imagery and real camera images used to train the GAN.

Motivation & Objective

• Motivate image forensics to counter GAN-generated imagery and online disinformation.
• Analyze GAN generator architecture to identify color-based cues that differentiate GAN outputs from real images.
• Propose practical detectors leveraging these cues that work with limited training data.
• Evaluate the detectors on benchmark datasets to assess discrimination between GAN and camera images.

Proposed method

• Examine how the GAN generator collapses depth layers to RGB channels and compare learned weights to camera spectral responses.
• Use r-vs-g chromaticity histograms with an INH-based classifier to detect color-formation cues.
• Compute saturation-based features measuring the frequency of over- and under-exposed pixels and train a linear SVM on these features.
• Apply the approach to two datasets (GAN Crop and GAN Full) from the NIST Media Forensics Challenge 2018.

Experimental results

Research questions

• RQ1 Can color-formation cues in GAN generators be exploited to distinguish GAN-generated imagery from real camera imagery?
• RQ2 Do normalization and multi-channel depth-to-RGB collapsing in GANs produce detectable saturation or color-statistics artifacts?
• RQ3 How effective are saturation-based and color-based forensics on benchmark GAN datasets in different manipulation scenarios?

Key findings

• Saturation-based features achieve notable discrimination, with AUC of 0.70 on fully GAN-generated images and 0.61 on GAN Full images.
• Color-based forensics perform near random, with AUCs of 0.56 and 0.54 on the two datasets.
• GAN normalization tends to suppress saturation, reducing exposure variability compared to real camera images.
• Color-channel formation in GANs uses learned weights that differ from real camera spectral sensitivities, enabling potential detection via color statistics.
• The color-forensics method shows limited effectiveness likely due to training data constraints and possible focus manipulation in some real images.