[Paper Review] Style Transfer Applied to Face Liveness Detection with User-Centered Models

A R Israel Laurensi, Luciana Trinkaus Menon+3 more|arXiv (Cornell University)|Jul 16, 2019

Face recognition and analysis27 references4 citations

TL;DR

This paper proposes a user-centered face anti-spoofing model (FAS-UCM) that generates spoof images via style transfer using VGG19 to augment training data, enabling subject-specific liveness detection. The method achieves an average classification error rate of 0.22 on the SiW database using a custom Spoof-ModNet architecture, demonstrating feasibility for personalized spoof detection despite limited real spoof data.

ABSTRACT

This paper proposes a face anti-spoofing user-centered model (FAS-UCM). The major difficulty, in this case, is obtaining fraudulent images from all users to train the models. To overcome this problem, the proposed method is divided in three main parts: generation of new spoof images, based on style transfer and spoof image representation models; training of a Convolutional Neural Network (CNN) for liveness detection; evaluation of the live and spoof testing images for each subject. The generalization of the CNN to perform style transfer has shown promising qualitative results. Preliminary results have shown that the proposed method is capable of distinguishing between live and spoof images on the SiW database, with an average classification error rate of 0.22.

Motivation & Objective

• Address the challenge of limited spoof image data for training user-specific face liveness detection models.
• Overcome the impracticality of collecting spoof images from every individual user in real-world scenarios.
• Develop a data augmentation strategy using style transfer to synthesize realistic spoof images from real face images.
• Train subject-specific convolutional neural networks (CNNs) to improve generalization and detection accuracy per user.
• Evaluate the performance of user-centered models on the SiW database using standard metrics like ACER, FAR, and FRR.

Proposed method

• Use the Gatys et al. style transfer method based on VGG19 to transfer the style of spoof images onto real face images, generating synthetic spoof samples.
• Select representative spoof images from the SiW database to serve as style references for transfer.
• Train two CNN architectures—Spoof-ModNet and MobileNetV2—on subject-specific data combining real and generated spoof images.
• Apply a user-centered training protocol where each subject’s model is trained and tested independently on their own data.
• Use perceptual loss and content loss to preserve facial identity while transferring spoofing artifacts (e.g., moiré patterns) during style transfer.
• Evaluate models using ACER, FAR, FRR, and F1 score, with testing performed per subject to simulate real-world deployment.

Experimental results

Research questions

• RQ1 Can style transfer techniques effectively generate realistic spoof face images from real images to augment training data for liveness detection?
• RQ2 How does user-centered training with synthetically generated spoof images affect the performance of face liveness detection models?
• RQ3 What is the impact of different CNN architectures (Spoof-ModNet vs. MobileNetV2) on the accuracy and robustness of user-specific liveness detection?
• RQ4 How does the proposed method generalize across subjects when only limited real spoof data is available?
• RQ5 Can the use of style transfer-based data augmentation outperform random classification in user-centered face liveness detection?

Key findings

• The proposed Spoof-ModNet achieved an average classification error rate (ACER) of 0.22 on the SiW database, outperforming MobileNetV2, which had an ACER of 0.26.
• The Spoof-ModNet achieved a 75% accuracy, 44% FAR, 1% FRR, and 78% F1 score, indicating strong performance in distinguishing live and spoof images.
• Over 50% of subjects achieved accuracy above 70% with Spoof-ModNet, while MobileNetV2 showed more consistent but lower performance, with 50% of subjects between 65.07% and 76.28% accuracy.
• The minimum accuracy for Spoof-ModNet was 34.69%, and the maximum was 99.49%, indicating high variability across subjects but strong upper performance.
• Despite higher ACER than state-of-the-art methods, the proposed approach significantly outperforms random classification and is feasible for personalized liveness detection.
• The method demonstrates that style transfer can generate plausible spoof images that improve model generalization when training user-centered models.