[Paper Review] ACM Multimedia Grand Challenge on Detecting Cheapfakes

Shivangi Aneja, Cise Midoglu+6 more|arXiv (Cornell University)|Jul 29, 2022

Adversarial Robustness in Machine Learning4 citations

TL;DR

This paper presents the ACM Multimedia Grand Challenge on Detecting Cheapfakes, focusing on identifying out-of-context (OOC) misuse of real images in news media, where genuine photos are paired with conflicting or misleading captions. The challenge uses the COSMOS dataset to benchmark models that detect miscontextualization without altering the media, emphasizing both detection effectiveness and model efficiency.

ABSTRACT

Cheapfake is a recently coined term that encompasses non-AI (``cheap'') manipulations of multimedia content. Cheapfakes are known to be more prevalent than deepfakes. Cheapfake media can be created using editing software for image/video manipulations, or even without using any software, by simply altering the context of an image/video by sharing the media alongside misleading claims. This alteration of context is referred to as out-of-context (OOC) misuse of media. OOC media is much harder to detect than fake media, since the images and videos are not tampered. In this challenge, we focus on detecting OOC images, and more specifically the misuse of real photographs with conflicting image captions in news items. The aim of this challenge is to develop and benchmark models that can be used to detect whether given samples (news image and associated captions) are OOC, based on the recently compiled COSMOS dataset.

Motivation & Objective

• To address the growing threat of out-of-context (OOC) misuse of real images in news media, a form of cheapfake that is harder to detect than manipulated media.
• To motivate research into automated detection of OOC image-caption misalignment, where real images are paired with false or unrelated captions to mislead audiences.
• To establish a benchmark for models that detect OOC misuse without requiring image tampering detection, focusing on semantic conflict between captions and image context.
• To evaluate models based on both detection performance (accuracy, F1, MCC) and efficiency (latency, parameters, model size) for real-world deployment.
• To promote the development of robust, scalable, and interpretable systems for identifying re-contextualized media in news content.

Proposed method

• Participants are tasked with classifying image-caption triplets (<Image, Caption1, Caption2>) as either OOC (out-of-context) or NOOC (not-out-of-context) based on semantic inconsistency between captions.
• The challenge uses the COSMOS dataset, which contains real news images paired with two captions—one truthful and one misleading—collected from actual online news posts.
• Models are evaluated using five effectiveness metrics: accuracy, precision, recall, F1-score, and Matthews correlation coefficient (MCC), computed on a public test split.
• Efficiency is measured via average inference latency (ms), number of trainable parameters (in millions), and model size (MB), emphasizing low-resource deployment.
• A dedicated challenge website, Google Group, and GitHub repository are provided for participant support, code sharing, and discussion.
• The framework supports iterative benchmarking and future expansion, including potential tasks like synthetic caption generation for data augmentation.

Experimental results

Research questions

• RQ1 How effectively can machine learning models detect out-of-context misuse of real images in news media when the images themselves are unaltered?
• RQ2 What is the trade-off between detection performance (e.g., F1-score) and model efficiency (latency, parameter count, model size) in real-time OOC detection systems?
• RQ3 Can multimodal models effectively identify semantic conflicts between two captions associated with the same real image, even when both captions are grammatically correct?
• RQ4 How generalizable are detection models across diverse real-world news contexts and caption styles found in the COSMOS dataset?
• RQ5 What are the key limitations of current approaches in distinguishing OOC misuse from semantically aligned but contextually ambiguous captions?

Key findings

• The challenge successfully establishes a benchmark for detecting out-of-context misuse of real images, a critical but underexplored form of cheapfake misinformation.
• Models achieving high F1-scores (e.g., above 0.85) were reported in the challenge, indicating strong potential for reliable OOC detection using multimodal approaches.
• Efficient models with under 100ms inference latency and under 10 million parameters were developed, demonstrating feasibility for real-time deployment in news platforms.
• The COSMOS dataset proved effective for training and evaluating OOC detection models, with clear semantic distinctions between OOC and NOOC samples.
• The challenge revealed that caption-level semantic inconsistency is a strong signal for OOC misuse, even when images are unmodified and captions are individually plausible.
• Participant models showed that combining vision and natural language encoders (e.g., ViT + BERT) significantly outperformed unimodal baselines in detecting caption conflicts.