[Paper Review] ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection

Davide Cozzolino, Justus Thies+4 more|arXiv (Cornell University)|Dec 6, 2018

Digital Media Forensic DetectionComputer Science57 references220 citations

TL;DR

ForensicTransfer introduces an autoencoder-based embedding that disentangles real/fake representations to achieve strong zero-shot and few-shot transfer to unseen forgery methods, outperforming state-of-the-art detectors with limited target-domain data.

ABSTRACT

Distinguishing manipulated from real images is becoming increasingly difficult as new sophisticated image forgery approaches come out by the day. Naive classification approaches based on Convolutional Neural Networks (CNNs) show excellent performance in detecting image manipulations when they are trained on a specific forgery method. However, on examples from unseen manipulation approaches, their performance drops significantly. To address this limitation in transferability, we introduce Forensic-Transfer (FT). We devise a learning-based forensic detector which adapts well to new domains, i.e., novel manipulation methods and can handle scenarios where only a handful of fake examples are available during training. To this end, we learn a forensic embedding based on a novel autoencoder-based architecture that can be used to distinguish between real and fake imagery. The learned embedding acts as a form of anomaly detector; namely, an image manipulated from an unseen method will be detected as fake provided it maps sufficiently far away from the cluster of real images. Comparing to prior works, FT shows significant improvements in transferability, which we demonstrate in a series of experiments on cutting-edge benchmarks. For instance, on unseen examples, we achieve up to 85% in terms of accuracy, and with only a handful of seen examples, our performance already reaches around 95%.

Motivation & Objective

• Address the rapid emergence of new image forgery methods by enhancing transferability of forgery detectors.
• Develop a weakly-supervised, autoencoder-based detector that disentangles real/fake representations in latent space.
• Enable zero-shot and few-shot adaptation to unseen manipulations with minimal labeled target data.
• Show that the proposed method achieves state-of-the-art transfer performance across diverse datasets.

Proposed method

• Propose an encoder–decoder (autoencoder) architecture that learns a forensic embedding for real vs forged images.
• Split the latent space into two disjoint parts h0 (real) and h1 (fake) and train to activate only the corresponding part for each class.
• Use a reconstruction loss L_REC to preserve information in latent space and an activation loss L_ACT to enforce class-specific activations (a0 or a1) in the latent space.
• Compute class activation a_c(x) as the L1 norm of the corresponding latent part, and classify by comparing a0(x) and a1(x).
• Preprocess inputs with a high-pass residual (third-order derivative) to emphasize forgery cues.
• Fine-tune the source-domain trained model on a small target-domain set to achieve rapid adaptation (few-shot/zero-shot).
• Train with Adam (lr=0.001), batch size 64, monitor validation loss, stop after 30 epochs without improvement.

Experimental results

Research questions

• RQ1 Can a latent-space disentangled autoencoder improve transferability of forgery detectors to unseen manipulation methods?
• RQ2 How few labeled target-domain samples are sufficient to achieve high accuracy on a new manipulation domain?
• RQ3 Does constraining latent activations per class reduce intra-class variance and improve cross-domain generalization?
• RQ4 What is the impact of residual-based preprocessing on transfer performance?
• RQ5 How does ForensicTransfer compare to existing CNN-based detectors under zero-shot and few-shot scenarios?

Key findings

• ForensicTransfer achieves high accuracy on unseen manipulations, with up to 85% accuracy in zero-shot transfers and around 95% with a handful of target-domain examples.
• Ablation studies show that removing residual preprocessing, reconstruction loss, or using cross-entropy loss degrades transfer performance, confirming design choices.
• The proposed activation-based loss reduces intra-class variance and improves generalization to new manipulation domains.
• Zero-shot and few-shot transfer outperform baseline CNN detectors across multiple source-target domain pairings.
• Multi-source training (many-to-one transfer) further improves target-domain detection, with gains over single-source setups, especially in low-shot regimes.