[论文解读] Boosting Masked Face Recognition with Multi-Task ArcFace

David Montero, Marcos Nieto+2 more|arXiv (Cornell University)|Apr 20, 2021

Face recognition and analysis参考文献 26被引用 7

一句话总结

本文提出多任务ArcFace（MTArcFace），一种多任务学习框架，通过修改ResNet-50并结合ArcFace损失与二元交叉熵损失，联合优化遮挡人脸识别与口罩使用分类。该方法在遮挡LFW数据集上达到98.92%的准确率，在口罩检测任务中平均准确率达99.78%，同时在非遮挡数据上保持接近原始性能。

ABSTRACT

In this paper, we address the problem of face recognition with masks. Given the global health crisis caused by COVID-19, mouth and nose-covering masks have become an essential everyday-clothing-accessory. This sanitary measure has put the state-of-the-art face recognition models on the ropes since they have not been designed to work with masked faces. In addition, the need has arisen for applications capable of detecting whether the subjects are wearing masks to control the spread of the virus. To overcome these problems a full training pipeline is presented based on the ArcFace work, with several modifications for the backbone and the loss function. From the original face-recognition dataset, a masked version is generated using data augmentation, and both datasets are combined during the training process. The selected network, based on ResNet-50, is modified to also output the probability of mask usage without adding any computational cost. Furthermore, the ArcFace loss is combined with the mask-usage classification loss, resulting in a new function named Multi-Task ArcFace (MTArcFace). Experimental results show that the proposed approach highly boosts the original model accuracy when dealing with masked faces, while preserving almost the same accuracy on the original non-masked datasets. Furthermore, it achieves an average accuracy of 99.78% in mask-usage classification.

研究动机与目标

• 解决新冠疫情期间口罩导致的人脸识别准确率显著下降的问题。
• 开发统一的深度学习框架，无需增加推理成本即可同时提升遮挡人脸识别性能与口罩使用检测能力。
• 在非遮挡人脸识别数据集上保持高性能，确保向后兼容性。
• 通过数据增强技术生成合成遮挡训练数据集，以增强模型泛化能力。

提出的方法

• 通过数据增强技术在原始人脸识别数据集中图像上合成添加口罩，以扩充数据集。
• 修改ResNet-50主干网络，使其在不增加计算成本的前提下输出深度特征嵌入与口罩使用概率预测。
• 提出一种新型多任务ArcFace（MTArcFace）损失函数，结合基于ArcFace边距的分类损失与二元交叉熵损失以实现口罩检测。
• 在原始人脸与遮挡人脸的联合数据集上端到端训练模型，以提升对遮挡的鲁棒性。
• 在两个任务间共享特征提取器，实现人脸识别与口罩检测的联合优化。
• 对口罩概率输出应用0.5的阈值以实现二元口罩使用分类。

实验结果

研究问题

• RQ1 单一深度学习模型能否在同时检测口罩使用的同时有效识别遮挡人脸？
• RQ2 在遮挡与非遮挡数据上联合训练，对遮挡与非遮挡人脸识别基准性能有何影响？
• RQ3 与原始ArcFace模型相比，该方法在非遮挡人脸上的准确率保留程度如何？
• RQ4 该模型在判断人脸是否佩戴口罩方面的准确率是多少？

主要发现

• 所提出的MTArcFace模型在遮挡LFW数据集上达到98.92%的验证准确率，显著优于原始ArcFace模型的94.75%。
• 在CFP_FP数据集上，模型将遮挡人脸识别准确率提升了近12%（从76.81%提升至88.43%）。
• 模型在非遮挡数据集上保持高性能，LFW数据集上的准确率仅下降0.17%（99.45% vs. 原始模型的99.62%）。
• 口罩使用分类任务在所有测试数据集上的平均准确率达99.78%，在CFP_FP数据集上表现最低，为98.82%。
• 模型在侧脸上的表现低于正面人脸，表明训练数据中需增加更多侧脸样本。
• 该方法有效平衡了双重目标：遮挡人脸识别与口罩检测，且对非遮挡识别性能的损失极小。