[Paper Review] Going Deeper Into Face Detection: A Survey

Shervin Minaee, Ping Luo+2 more|arXiv (Cornell University)|Mar 27, 2021

Face recognition and analysisComputer Science86 references51 citations

TL;DR

A comprehensive survey of deep-learning based face detection methods, architectures, datasets, benchmarks, and challenges, organized by model category.

ABSTRACT

Face detection is a crucial first step in many facial recognition and face analysis systems. Early approaches for face detection were mainly based on classifiers built on top of hand-crafted features extracted from local image regions, such as Haar Cascades and Histogram of Oriented Gradients. However, these approaches were not powerful enough to achieve a high accuracy on images of from uncontrolled environments. With the breakthrough work in image classification using deep neural networks in 2012, there has been a huge paradigm shift in face detection. Inspired by the rapid progress of deep learning in computer vision, many deep learning based frameworks have been proposed for face detection over the past few years, achieving significant improvements in accuracy. In this work, we provide a detailed overview of some of the most representative deep learning based face detection methods by grouping them into a few major categories, and present their core architectural designs and accuracies on popular benchmarks. We also describe some of the most popular face detection datasets. Finally, we discuss some current challenges in the field, and suggest potential future research directions.

Motivation & Objective

• Review the progress of deep-learning based face detection from the start of the deep learning era to the present.
• Categorize and compare major model families (Cascade-CNN, RCNN/Faster-RCNN, SSD, FPN, Transformers, and others).
• Summarize widely used face detection datasets and evaluation benchmarks.
• Discuss current challenges and propose potential directions for future research.

Proposed method

• Group existing deep face detection works into major categories based on architectural contributions.
• Provide overviews of backbones and core architectural ideas (CNNs, R-CNN, SSD, FPN, GANs, etc.).
• Summarize key models within each category and their main technical contributions.
• Outline popular benchmarks and evaluation metrics for deep face detection.
• Discuss challenges in unconstrained settings and suggest future research directions.

Experimental results

Research questions

• RQ1 What are the main deep-learning categories used for face detection and their core architectural ideas?
• RQ2 How do backbone architectures, loss functions, and training strategies influence performance on standard face detection benchmarks?
• RQ3 What datasets and evaluation metrics dominate current face detection research, and what gaps remain?
• RQ4 What are the primary challenges in face detection in the wild and potential avenues to address them?

Key findings

• The survey covers more than fifty deep face detection models across multiple architectural categories.
• It provides insight into training data, network backbones, loss functions, and training strategies underlying performance gains.
• It discusses popular benchmarks and metrics, as well as trends across datasets such as FDDB and Wider Face.
• It highlights challenges like variations in pose, scale, illumination, occlusion, and suggests directions including transformer-based approaches and context modeling.
• It emphasizes the evolution from Cascade-CNN and R-CNN based methods to single-shot detectors and feature pyramid networks, with attention to recent transformer-based ideas.