[Paper Review] BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs

Valentin Bazarevsky, Yury Kartynnik+3 more|arXiv (Cornell University)|Jul 11, 2019

Face recognition and analysis7 references248 citations

TL;DR

BlazeFace presents a lightweight face detector optimized for mobile GPU inference, achieving 200–1000+ FPS on flagship devices with a GPU-friendly SSD-like anchor scheme and a novel tie-resolution method. It also provides 6 facial keypoints for rotation-aware crops in AR pipelines.

ABSTRACT

We present BlazeFace, a lightweight and well-performing face detector tailored for mobile GPU inference. It runs at a speed of 200-1000+ FPS on flagship devices. This super-realtime performance enables it to be applied to any augmented reality pipeline that requires an accurate facial region of interest as an input for task-specific models, such as 2D/3D facial keypoint or geometry estimation, facial features or expression classification, and face region segmentation. Our contributions include a lightweight feature extraction network inspired by, but distinct from MobileNetV1/V2, a GPU-friendly anchor scheme modified from Single Shot MultiBox Detector (SSD), and an improved tie resolution strategy alternative to non-maximum suppression.

Motivation & Objective

• Develop a compact, GPU-friendly face detector for mobile devices optimized for AR pipelines.
• Increase inference speed while maintaining high face detection accuracy.
• Introduce architectural tweaks (anchors, tie resolution) to suit mobile GPUs and reduce jitter in video streams.
• Enable rotation-aware facial crops via 6 keypoints to improve downstream tasks.

Proposed method

• Design a lightweight feature extractor inspired by MobileNetV1/V2 but tailored for fast detection.
• Introduce a GPU-friendly anchor scheme that stops at 8x8 feature maps with 6 anchors per pixel at 8x8.
• Propose a tie-resolution strategy alternative to non-maximum suppression to stabilize overlapping predictions.
• Produce six facial keypoints (eye centers, ear tragions, mouth center, nose tip) for rotation estimation.
• Maintain a full-resolution 8x8 feature map to reduce anchor overlap-induced jitter and enable smoother temporal predictions.

Experimental results

Research questions

• RQ1 Can a compact CNN backbone and GPU-friendly anchors deliver real-time face detection on mobile GPUs?
• RQ2 Does a novel tie-resolution method improve stability over traditional NMS in dense anchor scenarios?
• RQ3 How does BlazeFace's accuracy and latency compare to MobileNetV2-SSD on mobile GPUs?
• RQ4 Can additional facial keypoints enable rotation-aware crops to improve downstream AR tasks?

Key findings

• BlazeFace achieves 98.61% average precision on frontal faces with 0.6 ms inference time on iPhone XS using TensorFlow Lite GPU in FP16.
• MobileNetV2-SSD achieves 97.95% AP with 2.1 ms inference time under the same framework.
• Across devices, BlazeFace significantly outperforms MobileNetV2-SSD in inference speed (e.g., iPhone XS: 0.6 ms vs 2.1 ms).
• The proposed tie-resolution strategy reduces temporal jitter by up to 40% on frontal and 30% on rear camera datasets.
• Regression parameter error for BlazeFace is 10.4% of inter-ocular distance (versus 7.4% for MobileNetV2-SSD), with a 5.3% jitter metric.