[Paper Review] RetinaFace: Single-stage Dense Face Localisation in the Wild

Jiankang Deng, Jia Guo+4 more|arXiv (Cornell University)|May 2, 2019

Face recognition and analysis73 references423 citations

TL;DR

RetinaFace introduces a one-stage dense face localization detector that jointly predicts face scores, boxes, five facial landmarks, and dense 3D face information, achieving state-of-the-art results on WIDER FACE hard and contributing to improved face recognition benchmarks.

ABSTRACT

Though tremendous strides have been made in uncontrolled face detection, accurate and efficient face localisation in the wild remains an open challenge. This paper presents a robust single-stage face detector, named RetinaFace, which performs pixel-wise face localisation on various scales of faces by taking advantages of joint extra-supervised and self-supervised multi-task learning. Specifically, We make contributions in the following five aspects: (1) We manually annotate five facial landmarks on the WIDER FACE dataset and observe significant improvement in hard face detection with the assistance of this extra supervision signal. (2) We further add a self-supervised mesh decoder branch for predicting a pixel-wise 3D shape face information in parallel with the existing supervised branches. (3) On the WIDER FACE hard test set, RetinaFace outperforms the state of the art average precision (AP) by 1.1% (achieving AP equal to 91.4%). (4) On the IJB-C test set, RetinaFace enables state of the art methods (ArcFace) to improve their results in face verification (TAR=89.59% for FAR=1e-6). (5) By employing light-weight backbone networks, RetinaFace can run real-time on a single CPU core for a VGA-resolution image. Extra annotations and code have been made available at: https://github.com/deepinsight/insightface/tree/master/RetinaFace.

Motivation & Objective

• Motivate robust, efficient face localization in the wild across a wide range of scales and occlusions.
• Propose a one-stage dense detector that leverages extra-supervised (facial landmarks) and self-supervised (dense 3D mesh) signals.
• Show that joint multi-task learning improves detection accuracy and downstream face recognition.
• Provide a lightweight, real-time option suitable for CPU and mobile deployment.
• Release annotations and code to facilitate further research.

Proposed method

• Use a single-stage detector with a feature pyramid (P2–P6) for multi-scale face localization.
• Add an extra-supervised branch predicting five facial landmarks and a dense regression branch predicting pixel-wise 3D face shape.
• Incorporate a dense regression mesh decoder based on graph convolutions and a differentiable 3D renderer with a pixel-wise regression loss (L_pixel).
• Optimize a multi-task loss per anchor combining classification, box regression, landmark regression, and dense pixel-wise regression.
• Employ deformable convolutions in context modules and anchor-based training with OHEM for class imbalance.
• Enable real-time inference with lightweight backbones (e.g., MobileNet-0.25) and provide scalable options for CPU/GPU.

Experimental results

Research questions

• RQ1 Can extra supervision from facial landmarks improve hard-face detection in a single-stage dense detector?
• RQ2 Does adding a self-supervised dense 3D mesh regression branch enhance pixel-wise face localization and overall detection accuracy?
• RQ3 What is the impact of joint landmark and dense regression on face detection performance across WIDER FACE subsets?
• RQ4 How does RetinaFace affect downstream face recognition when used for detection and alignment?
• RQ5 What are the real-time performance characteristics of RetinaFace on CPU and mobile hardware?

Key findings

• On WIDER FACE hard, RetinaFace achieves AP 91.4%, surpassing the prior state-of-the-art by 1.1 percentage points.
• On IJB-C, RetinaFace improves ArcFace verification to TAR 89.59% at FAR 1e-6.
• A light-weight RetinaFace with MobileNet-0.25 runs at real-time speeds: 60 FPS on VGA images on CPU and 16 FPS on ARM for 640×480.
• Five facial landmarks significantly improve hard-subset AP and mAP on WIDER FACE (landmark branch + 0.775% mAP gain).
• Dense regression alone may help Easy/Medium but can slightly hurt Hard; joint learning with landmarks yields further gains.
• Dense 3D regression is trained self-supervised and competes with supervised dense methods, showing benefits for detection and alignment.