[Paper Review] The Effect of Wearing a Mask on Face Recognition Performance: an Exploratory Study

Naser Damer, Jonas Henry Grebe+4 more|arXiv (Cornell University)|Jul 27, 2020

Face recognition and analysis14 references33 citations

TL;DR

The paper presents a specifically collected dataset to study how real-world face masks affect three face recognition systems (ArcFace, SphereFace, and a COTS solution) and finds a notable drop in genuine–imposter separability for masked probes, especially under additional illumination.

ABSTRACT

Face recognition has become essential in our daily lives as a convenient and contactless method of accurate identity verification. Process such as identity verification at automatic border control gates or the secure login to electronic devices are increasingly dependant on such technologies. The recent COVID-19 pandemic have increased the value of hygienic and contactless identity verification. However, the pandemic led to the wide use of face masks, essential to keep the pandemic under control. The effect of wearing a mask on face recognition in a collaborative environment is currently sensitive yet understudied issue. We address that by presenting a specifically collected database containing three session, each with three different capture instructions, to simulate realistic use cases. We further study the effect of masked face probes on the behaviour of three top-performing face recognition systems, two academic solutions and one commercial off-the-shelf (COTS) system.

Motivation & Objective

• Motivate evaluation of face recognition under health-related occlusion (masks) in collaborative settings.
• Provide a first version of a realistic masked-face database with three sessions to simulate login/identity verification scenarios.
• Analyze how masked probes affect top-performing academic and commercial face recognition systems.
• Quantify shifts in score distributions and verification metrics to guide future dataset and method development.

Proposed method

• Introduce a new dataset with three sessions per subject and three capture conditions (baseline, mask, and masked with illumination).
• Evaluate three face recognition solutions (ArcFace, SphereFace, and a Neurotechnology COTS) on masked vs. unmasked probes.
• Use MTCNN for detection/alignment and 512-d feature vectors; compare with Euclidean (ArcFace) and Cosine (SphereFace) distances.
• Report FTX and standard verification metrics (EER, FMR100, FMR1000, ZeroFMR) and ROC curves across experiments.

Experimental results

Research questions

• RQ1 How does wearing a mask affect the genuine and imposter score distributions in state-of-the-art face recognition systems?
• RQ2 To what extent does additional illumination interact with mask-wearing to impact recognition performance?
• RQ3 Do academic (ArcFace, SphereFace) and commercial (COTS) systems respond differently to masked probes in terms of verification metrics and score separability?

Key findings

• Masked probes shift genuine score distributions toward imposter distributions, reducing separability and degrading verification performance across ArcFace, SphereFace, and COTS.
• Additional illumination (mask with light) further worsens performance for masked probes, likely due to new reflection/shadow patterns.
• ArcFace and SphereFace show clear degradation in EER, FMR100, FMR1000, and ZeroFMR under masked probes; SphereFace degrades more than ArcFace.
• COTS remains near-perfect in verification metrics, but still shows a large shift in genuine scores, indicating changed decision boundaries under masks.
• FTX (failure to extract) increases with masks, indicating detection/feature extraction challenges in masked scenarios.