[Paper Review] Exploring Hypergraph Representation on Face Anti-spoofing Beyond 2D Attacks

Wei Hu, Gusi Te+3 more|arXiv (Cornell University)|Nov 28, 2018

Biometric Identification and Security30 references4 citations

TL;DR

This paper proposes Hypergraph Convolutional Neural Networks (HGCNN) for 3D face anti-spoofing, using compact hypergraph representations of facial key points to enable posture-invariant, computation-efficient feature learning. By leveraging RGB and depth modalities through hypergraph convolution and depth auxiliary supervision, the method achieves state-of-the-art performance on 2D and 3D anti-spoofing benchmarks, including a newly constructed 3D face attack database with color, depth, and infrared data.

ABSTRACT

Face anti-spoofing plays a crucial role in protecting face recognition systems from various attacks. Previous model-based and deep learning approaches achieve satisfactory performance for 2D face spoofs, but remain limited for more advanced 3D attacks such as vivid masks. In this paper, we address 3D face anti-spoofing via the proposed Hypergraph Convolutional Neural Networks (HGCNN). Firstly, we construct a computation-efficient and posture-invariant face representation with only a few key points on hypergraphs. The hypergraph representation is then fed into the designed HGCNN with hypergraph convolution for feature extraction, while the depth auxiliary is also exploited for 3D mask anti-spoofing. Further, we build a 3D face attack database with color, depth and infrared light information to overcome the deficiency of 3D face anti-spoofing data. Experiments show that our method achieves the state-of-the-art performance over widely used 3D and 2D databases as well as the proposed one under various tests.

Motivation & Objective

• Address the challenge of 3D mask attacks in face anti-spoofing, which remain underexplored despite progress in 2D attack detection.
• Overcome limitations of traditional 2D-based methods and simple graph models in capturing high-order facial relationships and posture variations.
• Develop a computation-efficient and robust facial representation using only key points on a hypergraph to reduce complexity while preserving discriminative features.
• Integrate depth information via hypergraph convolution to enhance detection of 3D masks, which are increasingly realistic and difficult to detect.
• Build a comprehensive 3D face attack database with color, depth, and infrared data to address the scarcity of diverse, high-quality 3D spoofing data.

Proposed method

• Construct a hypergraph representation using only 68 facial key points and interpolated points, modeling high-order relationships via hyperedges to ensure posture invariance.
• Design a Hypergraph Convolutional Neural Network (HGCNN) that applies hypergraph convolution to extract hierarchical features from both RGB and depth modalities.
• Apply the same hypergraph structure to both RGB and depth inputs, enabling joint feature learning while preserving modality-specific characteristics.
• Fuse depth and RGB features after hypergraph convolution, using depth as an auxiliary signal to improve 3D mask detection performance.
• Use a multi-modal input strategy with RGB and HSV color spaces to enhance texture representation, increasing input channels to 6 for richer feature learning.
• Optimize hypergraph construction by increasing hyperedge degrees to capture global facial patterns, improving robustness to sparse landmarking.

Experimental results

Research questions

• RQ1 Can hypergraph-based representation effectively model high-order facial relationships while remaining invariant to pose and expression variations?
• RQ2 To what extent does hypergraph convolution improve feature learning compared to standard graph convolution in face anti-spoofing?
• RQ3 How effective is the fusion of RGB and depth modalities through shared hypergraph structures in detecting 3D mask attacks?
• RQ4 Can a newly collected 3D face attack database with multi-spectral data (RGB, depth, IR) improve model generalization and performance on real-world spoofing attacks?
• RQ5 Does the proposed method achieve state-of-the-art performance across diverse benchmarks, including both 2D and 3D attack scenarios?

Key findings

• The proposed HGCNN achieves a 0.7% ACER on the FA3D dataset under intra-test protocol 1, outperforming existing methods on 3D mask detection.
• On the Oulu-NPU dataset, HGCNN (RGB+HSV) achieves 6.4% ACER, outperforming baseline methods like MixedFASNet and GRADIANT_extra, demonstrating strong 2D anti-spoofing capability.
• The model achieves 34.0% HTER and 43.0% EER in cross-dataset testing (FA3D-3DMAD), indicating strong generalization despite domain shift in illumination and depth variance.
• Visualization shows that after two layers of hypergraph convolution, feature distances between neighboring landmarks (e.g., around eyes and mouth) become significantly smaller, indicating improved feature clustering.
• The depth auxiliary improves performance on 3D mask attacks, with ablation studies confirming its critical role in distinguishing high-fidelity 3D masks from real faces.
• The method achieves 99.6% accuracy and 0.5% HTER on FA3D under subject variation, demonstrating robustness to subject-level diversity.