[Paper Review] Masked Face Recognition Dataset and Application

Zhongyuan Wang, Guangcheng Wang+12 more|arXiv (Cornell University)|Mar 20, 2020

Face recognition and analysis7 references229 citations

TL;DR

The paper introduces three masked-face datasets (MFDD, RMFRD, SMFRD) and a face-eye based multi-granularity model that reaches 95% accuracy on masked-face recognition.

ABSTRACT

In order to effectively prevent the spread of COVID-19 virus, almost everyone wears a mask during coronavirus epidemic. This almost makes conventional facial recognition technology ineffective in many cases, such as community access control, face access control, facial attendance, facial security checks at train stations, etc. Therefore, it is very urgent to improve the recognition performance of the existing face recognition technology on the masked faces. Most current advanced face recognition approaches are designed based on deep learning, which depend on a large number of face samples. However, at present, there are no publicly available masked face recognition datasets. To this end, this work proposes three types of masked face datasets, including Masked Face Detection Dataset (MFDD), Real-world Masked Face Recognition Dataset (RMFRD) and Simulated Masked Face Recognition Dataset (SMFRD). Among them, to the best of our knowledge, RMFRD is currently theworld's largest real-world masked face dataset. These datasets are freely available to industry and academia, based on which various applications on masked faces can be developed. The multi-granularity masked face recognition model we developed achieves 95% accuracy, exceeding the results reported by the industry. Our datasets are available at: https://github.com/X-zhangyang/Real-World-Masked-Face-Dataset.

Motivation & Objective

• Address the lack of publicly available masked-face datasets for deep learning-based recognition.
• Create datasets for masked-face detection and masked-face recognition (real-world and simulated).
• Develop a masked-face recognition model that leverages visible facial features (eyes, upper face) under mask occlusion.

Proposed method

• Construct MFDD by labeling and annotating crawled masked images for detection tasks.
• Build RMFRD by crawling public figures’ images and pairing masked/unmasked faces, followed by manual cleaning and annotation.
• Create SMFRD by automatically applying masks to large existing face datasets (LFW, WebFace) using a mask-wearing tool based on Dlib.
• Develop a multi-granularity recognition model that weights features from visible regions (eye, periocular, forehead, contours) to improve recognition under masking.
• Train and evaluate the model on the constructed datasets, achieving up to 95% accuracy.

Experimental results

Research questions

• RQ1 How can publicly available masked-face datasets be built to support deep learning for detection and recognition?
• RQ2 What dataset design (real-world vs simulated) yields better masked-face recognition performance?
• RQ3 Can a face-eye based multi-granularity model improve recognition accuracy when faces are partially occluded by masks?

Key findings

• MFDD contains 24,771 masked-face images suitable for detection training.
• RMFRD includes 5,000 masked images and 90,000 unmasked images of 525 individuals, claimed as the world’s largest real-world masked-face dataset.
• SMFRD provides up to 500,000 masked-face images of 10,000 subjects via automated mask application to LFW and WebFace.
• The proposed multi-granularity model improves masked-face recognition accuracy from baseline (~50%) to 95%.
• Masked-face recognition accuracy in industry ranges around 85-90% in real-world conditions, with the authors achieving 95% on their setup.