[Paper Review] In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking

Yuezun Li, Ming‐Ching Chang+1 more|arXiv (Cornell University)|Jun 7, 2018

Generative Adversarial Networks and Image SynthesisComputer Science17 references181 citations

TL;DR

The paper presents a forensic method that detects AI-generated fake face videos by analyzing eye blinking with a Long-term Recurrent Convolutional Network (LRCN) that leverages temporal eye dynamics to distinguish real from synthesized videos.

ABSTRACT

The new developments in deep generative networks have significantly improve the quality and efficiency in generating realistically-looking fake face videos. In this work, we describe a new method to expose fake face videos generated with neural networks. Our method is based on detection of eye blinking in the videos, which is a physiological signal that is not well presented in the synthesized fake videos. Our method is tested over benchmarks of eye-blinking detection datasets and also show promising performance on detecting videos generated with DeepFake.

Motivation & Objective

• Motivate the need for forensics in AI-generated fake face videos.
• Propose a physiological cue—eye blinking—as a reliable forensic signal.
• Develop a deep learning architecture that captures temporal eye dynamics to detect blinking absence in fakes.
• Evaluate the approach on eye-blinking datasets and DeepFake-generated videos.

Proposed method

• Detect faces per frame and align them to a canonical coordinate system.
• Crop eye regions based on landmarks to form eye sequences.
• Use a VGG16-based CNN to extract frame-level eye features, excluding final fully connected layers.
• Employ a Long-term Recurrent Convolutional Network (LRCN) with LSTM to model temporal eye-blink dynamics.
• Train front-end CNN on open/closed eye labels, then train LRCN end-to-end with back-propagation through time.

Experimental results

Research questions

• RQ1 Can the absence or irregularity of eye blinking in AI-generated videos serve as a detectable artifact?
• RQ2 Does incorporating temporal information via LRCN improve blinking-state classification over frame-wise CNNs or landmark-based methods?
• RQ3 How well does the eye-blinking-based method generalize to DeepFake-generated content?
• RQ4 What datasets and training strategies yield robust eye-blink detection in real versus fake videos?

Key findings

• LRCN achieves the best performance with an ROC of 0.99, outperforming CNN (0.98) and EAR (0.79).
• CNN alone captures eye state on individual frames but lacks temporal coherence.
• Modeling temporal dynamics via LSTM improves robustness to ambiguous frames and preserves smooth eye-state sequences.
• The method detects blinking patterns in authentic videos that are often missing or altered in DeepFake-generated videos.
• A dedicated Eye Blinking Video (EBV) dataset was created and used for evaluation, alongside CEW and other datasets.
• The approach demonstrates promising performance on detecting videos generated with DeepFake.