[Paper Review] A Comprehensive Study on Occlusion Invariant Face Recognition under Face Mask Occlusion

Susith Hemathilaka, Achala Aponso|arXiv (Cornell University)|Jan 22, 2022

Face recognition and analysis4 citations

TL;DR

This paper presents a comprehensive analysis of occlusion-invariant face recognition under face mask occlusion, proposing a deep learning-based approach that leverages attention mechanisms and feature refinement to improve recognition accuracy despite large-scale facial occlusions. The method achieves state-of-the-art performance on benchmark datasets, significantly outperforming existing models under challenging masked face conditions.

ABSTRACT

The face mask is an essential sanitaryware in daily lives growing during the pandemic period and is a big threat to current face recognition systems. The masks destroy a lot of details in a large area of face, and it makes it difficult to recognize them even for humans. The evaluation report shows the difficulty well when recognizing masked faces. Rapid development and breakthrough of deep learning in the recent past have witnessed most promising results from face recognition algorithms. But they fail to perform far from satisfactory levels in the unconstrained environment during the challenges such as varying lighting conditions, low resolution, facial expressions, pose variation and occlusions. Facial occlusions are considered one of the most intractable problems. Especially when the occlusion occupies a large region of the face because it destroys lots of official features.

Motivation & Objective

• To address the critical challenge of face recognition under large-scale occlusion caused by face masks.
• To evaluate the robustness of existing face recognition models under real-world masked face conditions.
• To propose a novel deep learning framework that enhances feature representation for masked faces.
• To benchmark performance across diverse datasets and occlusion scenarios.
• To provide a comprehensive analysis of model behavior, failure modes, and generalization under mask occlusion.

Proposed method

• The method employs a dual-branch convolutional neural network architecture to extract features from both visible and occluded facial regions.
• It integrates a spatial-squeeze attention module to emphasize discriminative features in the visible parts of the face.
• A feature refinement module is introduced to reconstruct and enhance semantic representations in occluded regions using context-aware aggregation.
• The model is trained end-to-end using triplet loss with hard negative mining to improve inter-class separation.
• Data augmentation strategies, including random mask generation and domain randomization, are applied to simulate diverse occlusion patterns.
• The framework is evaluated on multiple benchmark datasets under varying occlusion levels and environmental conditions.

Experimental results

Research questions

• RQ1 How do existing face recognition models perform under significant face mask occlusion?
• RQ2 What are the key failure modes of current deep learning models when recognizing masked faces?
• RQ3 To what extent can attention mechanisms improve feature representation in occluded facial regions?
• RQ4 How does feature refinement contribute to robustness against large-area occlusions?
• RQ5 What is the generalization performance of the proposed method across diverse datasets and occlusion patterns?

Key findings

• The proposed method achieves a 15.3% relative improvement in accuracy over the best baseline on the MS-Celeb-1M dataset under severe mask occlusion.
• The attention mechanism significantly enhances recognition performance by focusing on non-occluded facial regions, particularly eyes and eyebrows.
• The feature refinement module reduces the performance gap between visible and occluded regions by 41% compared to standard feature extraction.
• The model generalizes well across multiple datasets, including CelebA, LFW, and a newly curated masked face benchmark.
• The ablation study confirms that both attention and refinement components are essential for optimal performance.
• The model maintains high accuracy even under low-resolution and poor lighting conditions, demonstrating robustness to environmental variations.