[论文解读] Masked Face Recognition using ResNet-50

Bishwas Mandal, Adaeze Okeukwu+1 more|arXiv (Cornell University)|Apr 19, 2021

Face recognition and analysis参考文献 18被引用 70

一句话总结

本论文对预训练的 ResNet-50 模型进行微调以用于口罩面部识别，在 RMFRD 数据上对未遮挡人脸约达到 89.7% 的准确率，对口罩人脸约达到 47.9% 的准确率。

ABSTRACT

Over the last twenty years, there have seen several outbreaks of different coronavirus diseases across the world. These outbreaks often led to respiratory tract diseases and have proved to be fatal sometimes. Currently, we are facing an elusive health crisis with the emergence of COVID-19 disease of the coronavirus family. One of the modes of transmission of COVID- 19 is airborne transmission. This transmission occurs as humans breathe in the droplets released by an infected person through breathing, speaking, singing, coughing, or sneezing. Hence, public health officials have mandated the use of face masks which can reduce disease transmission by 65%. For face recognition programs, commonly used for security verification purposes, the use of face mask presents an arduous challenge since these programs were typically trained with human faces devoid of masks but now due to the onset of Covid-19 pandemic, they are forced to identify faces with masks. Hence, this paper investigates the same problem by developing a deep learning based model capable of accurately identifying people with face-masks. In this paper, the authors train a ResNet-50 based architecture that performs well at recognizing masked faces. The outcome of this study could be seamlessly integrated into existing face recognition programs that are designed to detect faces for security verification purposes.

研究动机与目标

• 研究在因 COVID-19 而被口罩遮挡时识别身份的挑战。
• 评估使用预训练的 ResNet-50 进行迁移学习是否能适应口罩面部识别。
• 评估遮挡对识别性能的影响，并确定有效的训练策略。
• 提供针对口罩面部识别的详细超参数调整和结构设计考虑。

提出的方法

• 通过在未遮挡人脸上微调预训练的 ResNet-50 模型来应用迁移学习。
• 在口罩面部数据上评估微调后的模型，以评估在遮挡下的识别能力。
• 尝试数据增强（随机水平翻转）和将数据集预处理为 180x180 输入。
• 在批量大小、优化器、 dropout、学习率和训练轮数上进行超参数调优以最大化性能。
• 使用 RMFRD 真实世界口罩面部数据集，70/30 训练/验证划分，77 个类别满足每类八张图像的条件。

实验结果

研究问题

• RQ1 基于 ResNet-50 的模型在未遮挡人脸上预训练后，是否能够在面部被遮挡时准确识别个体？
• RQ2 在使用迁移学习和超参数调优的情况下，未遮挡和遮挡人脸的性能有何差异？
• RQ3 哪些数据和训练策略（数据增强、裁剪、领域自适应）会影响口罩面部识别的性能？
• RQ4 在 ResNet-50 的口罩面部识别中，最优的超参数（优化器、批量大小、 dropout、学习率、训练轮数）是什么？

主要发现

Type of Data	Precision	Recall	F1-Score
unmasked	0.8933	0.8970	0.897
masked	0.4613	0.4719	0.4473

• 在未遮挡人脸上对预训练的 ResNet-50 进行微调，得到的准确率为 89.7016%，精确度 0.8993，召回率 0.8970，F1 0.897。
• 在口罩人脸上，最佳结果是 47.91% 的准确率，精确度 0.4613，召回 0.4719，F1 0.4473。
• 数据不平衡（未遮挡图像远多于口罩图像）和遮挡会降低口罩面部识别的性能，相较于未遮挡人脸识别。
• 超参数调优（将优化器从 SGD 改为 Adam、调整批量大小、 dropout 和学习率）将口罩面部准确度从约 21% 提高到 ~44.73%（以 F1 表示）。
• 研究结论认为遮挡对面部识别的 CNNs 构成显著挑战，并提出未来的技术，如使用带有模拟口罩的数据增强和领域自适应。