[Paper Review] Taming Self-Supervised Learning for Presentation Attack Detection: In-Image De-Folding and Out-of-Image De-Mixing.

Haozhe Liu, Zhe Kong+4 more|arXiv (Cornell University)|Sep 9, 2021

Digital Media Forensic Detection46 references4 citations

TL;DR

This paper proposes IF-OM, a self-supervised learning method for presentation attack detection (PAD) that uses in-image de-folding (local feature learning via cycle consistency) and out-of-image de-mixing (global instance-specific feature learning via topological consistency) to improve generalization against unknown presentation attack instruments. It achieves 18.60% EER on OULU-NPU and MSU-MFSD, outperforming baselines by 9.54% in EER reduction when trained on CASIA-FASD and Idiap Replay-Attack.

ABSTRACT

Biometric systems are vulnerable to the Presentation Attacks (PA) performed using various Presentation Attack Instruments (PAIs). Even though there are numerous Presentation Attack Detection (PAD) techniques based on both deep learning and hand-crafted features, the generalization of PAD for unknown PAI is still a challenging problem. The common problem with existing deep learning-based PAD techniques is that they may struggle with local optima, resulting in weak generalization against different PAs. In this work, we propose to use self-supervised learning to find a reasonable initialization against local trap, so as to improve the generalization ability in detecting PAs on the biometric system.The proposed method, denoted as IF-OM, is based on a global-local view coupled with De-Folding and De-Mixing to derive the task-specific representation for PAD.During De-Folding, the proposed technique will learn region-specific features to represent samples in a local pattern by explicitly maximizing cycle consistency. While, De-Mixing drives detectors to obtain the instance-specific features with global information for more comprehensive representation by maximizing topological consistency. Extensive experimental results show that the proposed method can achieve significant improvements in terms of both face and fingerprint PAD in more complicated and hybrid datasets, when compared with the state-of-the-art methods. Specifically, when training in CASIA-FASD and Idiap Replay-Attack, the proposed method can achieve 18.60% Equal Error Rate (EER) in OULU-NPU and MSU-MFSD, exceeding baseline performance by 9.54%. Code will be made publicly available.

Motivation & Objective

• Address the challenge of generalization in presentation attack detection (PAD) for unknown presentation attack instruments (PAIs).
• Overcome the limitation of deep learning-based PAD methods that often get trapped in local optima, reducing generalization performance.
• Improve representation learning for PAD by leveraging self-supervised pre-training to avoid poor local minima.
• Develop a unified framework that captures both local (region-specific) and global (instance-specific) features for robust PAD.
• Demonstrate superior performance on complex and hybrid datasets, including OULU-NPU and MSU-MFSD, under cross-dataset evaluation.

Proposed method

• Propose a global-local contrastive learning framework that combines in-image de-folding and out-of-image de-mixing for self-supervised representation learning.
• Implement de-folding by maximizing cycle consistency across cropped image regions to learn local, region-specific features.
• Implement de-mixing by maximizing topological consistency across different augmentations to learn global, instance-specific features.
• Use the self-supervised features as a strong initialization for downstream PAD classification, reducing risk of local optima.
• Train the final PAD classifier on the self-supervised features using standard cross-entropy loss with end-to-end fine-tuning.
• Leverage both local and global representations to improve robustness to diverse and unseen PAIs in real-world biometric systems.

Experimental results

Research questions

• RQ1 Can self-supervised learning with de-folding and de-mixing improve generalization in PAD for unknown presentation attack instruments?
• RQ2 How effective is the proposed global-local contrastive learning framework in capturing discriminative features for PAD across diverse datasets?
• RQ3 To what extent does the self-supervised initialization reduce the risk of local optima in deep learning-based PAD models?
• RQ4 How does the IF-OM method compare to state-of-the-art PAD methods in cross-dataset evaluation on OULU-NPU and MSU-MFSD?
• RQ5 Can the combination of in-image de-folding and out-of-image de-mixing yield better generalization than existing self-supervised or supervised approaches?

Key findings

• The proposed IF-OM method achieves 18.60% Equal Error Rate (EER) on OULU-NPU and MSU-MFSD when trained on CASIA-FASD and Idiap Replay-Attack, significantly outperforming baseline methods.
• The EER improvement over the baseline is 9.54 percentage points, demonstrating strong generalization to unseen PAIs and complex datasets.
• The method shows consistent performance gains across both face and fingerprint PAD tasks, indicating broad applicability.
• The use of self-supervised pre-training with cycle and topological consistency reduces the risk of local optima, enhancing model robustness.
• The de-folding and de-mixing components effectively capture local texture patterns and global structural information, improving detection accuracy.
• The code will be made publicly available, supporting reproducibility and further research in self-supervised PAD.