[论文解读] PFLD: A Practical Facial Landmark Detector

Xiaojie Guo, Siyuan Li+6 more|ArXiv.org|Feb 28, 2019

Face recognition and analysis参考文献 46被引用 35

一句话总结

PFLD 提供一个轻量级的端到端单阶段人脸关键点检测器，带有几何正则化损失和一个辅助姿态分支，在超快的移动端速度（超过 140 fps）和一个微小的 2.1 MB 模型下实现高精度。

ABSTRACT

Being accurate, efficient, and compact is essential to a facial landmark detector for practical use. To simultaneously consider the three concerns, this paper investigates a neat model with promising detection accuracy under wild environments e.g., unconstrained pose, expression, lighting, and occlusion conditions) and super real-time speed on a mobile device. More concretely, we customize an end-to-end single stage network associated with acceleration techniques. During the training phase, for each sample, rotation information is estimated for geometrically regularizing landmark localization, which is then NOT involved in the testing phase. A novel loss is designed to, besides considering the geometrical regularization, mitigate the issue of data imbalance by adjusting weights of samples to different states, such as large pose, extreme lighting, and occlusion, in the training set. Extensive experiments are conducted to demonstrate the efficacy of our design and reveal its superior performance over state-of-the-art alternatives on widely-adopted challenging benchmarks, i.e., 300W (including iBUG, LFPW, AFW, HELEN, and XM2VTS) and AFLW. Our model can be merely 2.1Mb of size and reach over 140 fps per face on a mobile phone (Qualcomm ARM 845 processor) with high precision, making it attractive for large-scale or real-time applications. We have made our practical system based on PFLD 0.25X model publicly available at \url{http://sites.google.com/view/xjguo/fld} for encouraging comparisons and improvements from the community.

研究动机与目标

• 满足在不受约束的环境中（姿态、照明、遮挡）实现高精度、高效且紧凑的人脸关键点检测的实际需求。
• 开发一个适用于移动/嵌入式设备的单阶段网络以实现所需速度。
• 在训练中融入几何正则化和数据不平衡处理，以提升在不同姿态和条件下的鲁棒性。
• 实现高度紧凑的模型，同时不牺牲最先进的精度，从而在移动硬件上实现实时部署。

提出的方法

• 设计一个使用 MobileNet 模块的端到端单阶段骨干网络，以减小尺寸并提高速度。
• 引入一个辅助网络来估计3D头部姿态（偏航、俯仰、滚转），无需额外注释。
• 提出一种新颖的损失函数，将基于3D姿态的几何正则化与样本权重相结合，以解决数据不平衡问题。
• 引入多尺度全连接（MS-FC）层以增大感受野，从而实现更精准的关键点定位。
• 使用姿态感知的增强与正则化进行训练；测试阶段仅使用骨干网络以提高效率。

实验结果

研究问题

• RQ1 在野外条件下（姿态、表情、照明、遮挡）以紧凑模型实现高精度的关键点定位有哪些途径？
• RQ2 几何正则化、姿态感知训练目标是否能提升相对于现有方法的鲁棒性？
• RQ3 在不牺牲精度的前提下，是否有可能在移动设备上维持小型化且实时的性能？

主要发现

• PFLD 的变体在 300W 和 AFLW 基准测试上优于若干现有方法。
• 0.25X 模型在显著减小尺寸（2.1 MB）和高速度下实现了较强的精度。
• 1X 模型在实现更高精度的同时，仍可在移动硬件上实现实时性能。
• 0.25X 与 1X 模型在 Qualcomm ARM 845 处理器上（按人脸计算）运行超过 140 fps。
• 辅助姿态分支和所提出的损失共同提升对姿态、遮挡和光照变化的鲁棒性。