[Paper Review] The Effect of Wearing a Face Mask on Face Image Quality

Biying Fu, Florian Kirchbuchner+1 more|arXiv (Cornell University)|Oct 21, 2021

Face recognition and analysis33 references4 citations

TL;DR

This study investigates how face masks degrade face image quality using state-of-the-art face image quality assessment (FIQA) methods, comparing real masks, simulated masks, and no-masked faces. It finds that masks significantly reduce image quality and verification performance, with simulated masks showing inconsistent effects compared to real masks, and reveals that FIQA networks focus less on masked regions, especially in synthetic masks, due to texture and color artifacts.

ABSTRACT

Due to the COVID-19 situation, face masks have become a main part of our daily life. Wearing mouth-and-nose protection has been made a mandate in many public places, to prevent the spread of the COVID-19 virus. However, face masks affect the performance of face recognition, since a large area of the face is covered. The effect of wearing a face mask on the different components of the face recognition system in a collaborative environment is a problem that is still to be fully studied. This work studies, for the first time, the effect of wearing a face mask on face image quality by utilising state-of-the-art face image quality assessment methods of different natures. This aims at providing better understanding on the effect of face masks on the operation of face recognition as a whole system. In addition, we further studied the effect of simulated masks on face image utility in comparison to real face masks. We discuss the correlation between the mask effect on face image quality and that on the face verification performance by automatic systems and human experts, indicating a consistent trend between both factors. The evaluation is conducted on the database containing (1) no-masked faces, (2) real face masks, and (3) simulated face masks, by synthetically generating digital facial masks on no-masked faces. Finally, a visual interpretation of the face areas contributing to the quality score of a selected set of quality assessment methods is provided to give a deeper insight into the difference of network decisions in masked and non-masked faces, among other variations.

Motivation & Objective

• To understand the impact of face masks on face image quality, a critical but understudied factor in face recognition performance degradation.
• To compare the effect of real masks versus simulated masks on face image quality and verification performance.
• To analyze whether face image quality assessment (FIQA) methods correlate with both automated face recognition and human expert performance.
• To visualize and interpret the attention mechanisms of FIQA networks on masked and non-masked faces to understand decision-making differences.
• To evaluate how mask color and shape variations in simulated masks affect FIQA scores and network attention patterns.

Proposed method

• Utilized four state-of-the-art face image quality assessment (FIQA) methods: MagFace, FaceQnet, and two others (not named) to quantify image quality under different mask conditions.
• Constructed a controlled database with three categories: no-masked faces (No-M), real face masks (Real-M), and simulated face masks (Sim-M) applied to the same base images.
• Applied simulated masks using standardized types (A–F) and colors as defined in prior work [27], including random color variations for Type C masks.
• Used Score-CAM visualization to interpret attention maps of FIQA networks, identifying which facial regions contribute most to quality scores.
• Evaluated face verification performance using a COTS FR system and human experts, comparing results with FIQA scores.
• Conducted controlled experiments to assess the correlation between FIQA scores, verification accuracy, and human judgment across real and simulated masks.

Experimental results

Research questions

• RQ1 How does wearing a face mask affect face image quality as measured by state-of-the-art FIQA methods?
• RQ2 Does the degradation in face image quality caused by masks correspond to the observed decline in face verification performance by automated systems and human experts?
• RQ3 How do simulated masks compare to real masks in terms of their effect on face image quality and verification performance?
• RQ4 Does the effect of simulated masks on image quality correlate with their effect on face verification performance, and how does this compare to real masks?
• RQ5 Which facial regions do FIQA networks attend to most, and how does this attention shift when processing real versus simulated masks with varying colors or shapes?

Key findings

• Wearing a face mask significantly reduces face image quality across all four FIQA methods, indicating a consistent degradation in image utility for recognition tasks.
• The drop in face image quality due to masks strongly correlates with the decline in face verification performance by both automated systems and human experts.
• Simulated masks do not fully replicate the quality degradation caused by real masks, suggesting that synthetic mask application does not fully capture real-world visual and textural effects.
• Mask color variations in simulated masks influence FIQA network attention and scores, indicating that neural network operations are sensitive to pixel-level changes even in non-visible regions.
• FIQA networks show reduced attention on masked facial regions—especially the nose and lower face—particularly in simulated masks, suggesting a loss of dependency on covered areas.
• The attention patterns of FIQA models differ between real and simulated masks, with real masks eliciting more nuanced attention on visible facial structures despite occlusion, while simulated masks often draw attention only to visible regions, ignoring subtle texture cues.