[Paper Review] Hybrid Deepfake Detection Utilizing MLP and LSTM

Jacob Mallet, Natalie Krueger+2 more|arXiv (Cornell University)|Apr 21, 2023

Digital Media Forensic Detection4 citations

TL;DR

This paper proposes a hybrid deepfake detection model combining Long Short-Term Memory (LSTM) and Multilayer Perceptron (MLP) networks to detect synthetic faces in images. Trained on the 140k Real and Fake Faces dataset, the model achieves a detection accuracy of up to 74.7%, demonstrating the effectiveness of combining sequential modeling with dense feedforward processing for deepfake identification.

ABSTRACT

The growing reliance of society on social media for authentic information has done nothing but increase over the past years. This has only raised the potential consequences of the spread of misinformation. One of the growing methods in popularity is to deceive users using a deepfake. A deepfake is an invention that has come with the latest technological advancements, which enables nefarious online users to replace their face with a computer generated, synthetic face of numerous powerful members of society. Deepfake images and videos now provide the means to mimic important political and cultural figures to spread massive amounts of false information. Models that can detect these deepfakes to prevent the spread of misinformation are now of tremendous necessity. In this paper, we propose a new deepfake detection schema utilizing two deep learning algorithms: long short term memory and multilayer perceptron. We evaluate our model using a publicly available dataset named 140k Real and Fake Faces to detect images altered by a deepfake with accuracies achieved as high as 74.7%

Motivation & Objective

• To address the rising threat of deepfake misinformation on social media platforms.
• To develop a robust deepfake detection system capable of identifying synthetic faces in images.
• To evaluate the performance of a hybrid model combining LSTM and MLP architectures for improved detection accuracy.
• To contribute a novel deep learning approach that leverages both temporal and spatial feature extraction for deepfake detection.

Proposed method

• The model employs an LSTM network to capture sequential patterns in facial features across image frames or patches.
• A Multilayer Perceptron (MLP) is used to learn high-level, non-linear representations from extracted image features.
• Features are extracted from the 140k Real and Fake Faces dataset, which contains both authentic and deepfake-generated facial images.
• The outputs from the LSTM and MLP are fused to produce a final prediction on whether an input image is real or fake.
• The model is trained using standard backpropagation with categorical cross-entropy loss and Adam optimization.
• Data augmentation and normalization techniques are applied to improve generalization and training stability.

Experimental results

Research questions

• RQ1 Can a hybrid architecture combining LSTM and MLP improve deepfake detection accuracy compared to standalone models?
• RQ2 How effective is the fusion of sequential (LSTM) and dense (MLP) modeling in detecting subtle artifacts in deepfake images?
• RQ3 What level of performance can be achieved using the 140k Real and Fake Faces dataset with this hybrid approach?
• RQ4 Does the integration of temporal modeling with feedforward processing enhance detection of synthetic facial features?

Key findings

• The hybrid LSTM-MLP model achieved a detection accuracy of up to 74.7% on the 140k Real and Fake Faces dataset.
• The integration of LSTM and MLP components improved detection performance compared to using either model in isolation.
• The model demonstrated the ability to learn complex, non-linear patterns in facial features indicative of deepfake generation.
• The results indicate that combining sequential and dense neural network architectures enhances the detection of synthetic facial content.