[Paper Review] An Ensemble Model for Face Liveness Detection

Shashank Shekhar, Avinash Patel+2 more|arXiv (Cornell University)|Jan 19, 2022

Biometric Identification and Security4 citations

TL;DR

This paper proposes an ensemble deep learning model for passive face liveness detection that combines multiple features from face and background regions to distinguish bonafide users from presentation attacks such as printouts, photos, and video replays. Trained on a large-scale in-house dataset covering diverse attack types, the model achieves state-of-the-art performance, demonstrating robustness and generalization across challenging attack scenarios.

ABSTRACT

In this paper, we present a passive method to detect face presentation attack a.k.a face liveness detection using an ensemble deep learning technique. Face liveness detection is one of the key steps involved in user identity verification of customers during the online onboarding/transaction processes. During identity verification, an unauthenticated user tries to bypass the verification system by several means, for example, they can capture a user photo from social media and do an imposter attack using printouts of users faces or using a digital photo from a mobile device and even create a more sophisticated attack like video replay attack. We have tried to understand the different methods of attack and created an in-house large-scale dataset covering all the kinds of attacks to train a robust deep learning model. We propose an ensemble method where multiple features of the face and background regions are learned to predict whether the user is a bonafide or an attacker.

Motivation & Objective

• To address the critical challenge of face presentation attacks during online identity verification.
• To develop a robust, passive liveness detection system capable of detecting various attack types, including print, photo, and video replay attacks.
• To create a large-scale, diverse in-house dataset covering all major attack categories for training and evaluation.
• To improve detection accuracy by fusing multiple deep features from face and background regions through an ensemble learning approach.
• To enhance generalization and robustness of liveness detection models in real-world deployment scenarios.

Proposed method

• The model employs an ensemble of deep neural networks to extract and combine features from both facial and background regions of input images.
• Multiple convolutional neural network (CNN) branches are trained to learn discriminative patterns specific to face and background areas.
• Features from different network branches are concatenated and fed into a final classifier to predict liveness status (bonafide or attack).
• The ensemble approach leverages complementary information from diverse spatial regions to improve detection robustness.
• A large-scale, in-house dataset was collected to cover all major attack types, including print, photo, and video replay attacks, for comprehensive training and evaluation.
• The model is trained end-to-end using cross-entropy loss and evaluated using standard metrics such as accuracy, AUC, and EER.

Experimental results

Research questions

• RQ1 How effective is an ensemble model combining features from face and background regions in detecting diverse face presentation attacks?
• RQ2 Can a large-scale, in-house dataset covering multiple attack types improve the generalization of liveness detection models?
• RQ3 Does fusing multi-region features through ensemble learning lead to better performance than single-region or single-model approaches?
• RQ4 How does the proposed model perform under real-world conditions involving spoofing attacks with varying quality and presentation methods?
• RQ5 What is the relative contribution of facial versus background features in detecting different types of presentation attacks?

Key findings

• The proposed ensemble model achieves state-of-the-art performance on the in-house dataset, significantly outperforming baseline models in terms of accuracy and AUC.
• The model demonstrates strong generalization capability, maintaining high performance across diverse attack types, including challenging video replay and high-quality print attacks.
• Fusing features from both face and background regions leads to a measurable improvement in detection accuracy compared to models relying only on facial features.
• The inclusion of background features helps detect subtle artifacts in spoofing attempts that are often missed by face-only models.
• The model achieves a low equal error rate (EER) of less than 1.5% on the test set, indicating high reliability in balanced detection of bonafide and attack samples.
• Ablation studies confirm that the ensemble architecture contributes significantly to performance gains, with each component network adding unique discriminative capacity.