[Paper Review] ASFD: Automatic and Scalable Face Detector

Jian Li, Bin Zhang+7 more|arXiv (Cornell University)|Jan 26, 2022

Face recognition and analysis55 references14 citations

TL;DR

This paper proposes ASFD, an automatic and scalable face detector that uses differentiable neural architecture search to discover a domain-specific Feature Aggregation and Enhancement (AutoFAE) module tailored for face detection. By addressing the domain gap between generic object and face detection, ASFD achieves state-of-the-art performance-efficiency trade-offs, with ASFD-D6 achieving 96.7/96.2/92.1 AP on WIDER Face test and ASFD-D0 running at over 320 FPS on V100 GPU with 3.1ms inference time.

ABSTRACT

Along with current multi-scale based detectors, Feature Aggregation and Enhancement (FAE) modules have shown superior performance gains for cutting-edge object detection. However, these hand-crafted FAE modules show inconsistent improvements on face detection, which is mainly due to the significant distribution difference between its training and applying corpus, COCO vs. WIDER Face. To tackle this problem, we essentially analyse the effect of data distribution, and consequently propose to search an effective FAE architecture, termed AutoFAE by a differentiable architecture search, which outperforms all existing FAE modules in face detection with a considerable margin. Upon the found AutoFAE and existing backbones, a supernet is further built and trained, which automatically obtains a family of detectors under the different complexity constraints. Extensive experiments conducted on popular benchmarks, WIDER Face and FDDB, demonstrate the state-of-the-art performance-efficiency trade-off for the proposed automatic and scalable face detector (ASFD) family. In particular, our strong ASFD-D6 outperforms the best competitor with AP 96.7/96.2/92.1 on WIDER Face test, and the lightweight ASFD-D0 costs about 3.1 ms, more than 320 FPS, on the V100 GPU with VGA-resolution images.

Motivation & Objective

• To address the performance degradation of existing hand-crafted Feature Aggregation and Enhancement (FAE) modules when applied to face detection due to domain distribution differences.
• To design a search space and search strategy specifically tailored for face detection, accounting for small-scale and crowded faces.
• To automatically discover an optimal FAE architecture (AutoFAE) via differentiable architecture search that outperforms existing modules in the face detection domain.
• To build a supernet based on AutoFAE and ResNet backbones to enable automatic model scaling for diverse efficiency constraints.
• To achieve a new state-of-the-art performance-efficiency trade-off across multiple face detection benchmarks.

Proposed method

• Propose a domain-aware search space for FAE modules, focusing on cross-scale and similar-scale connections to handle small and crowded faces.
• Apply bi-level optimization in differentiable architecture search to discover AutoFAE, which learns sparse, top-down connections and adaptive operations across pyramid layers.
• Integrate AutoFAE into a supernet with ResNet backbones and use genetic algorithm to search single-path architectures under latency constraints.
• Train the supernet and derived models using multi-scale inference and standard anchor-based head with multi-task loss.
• Replace standard FPN with AutoFAE in RetinaNet, FCOS, and Faster R-CNN to evaluate generalization on COCO.
• Use cumulative distribution functions to analyze data distribution differences between COCO and WIDER Face, justifying the need for domain-specific FAE design.

Experimental results

Research questions

• RQ1 Why do existing hand-crafted FAE modules fail to generalize to face detection despite their success in generic object detection?
• RQ2 How can a neural architecture search be adapted to discover FAE modules specifically optimized for the unique challenges of face detection, such as small and crowded faces?
• RQ3 Can an automatically searched FAE module (AutoFAE) outperform hand-crafted FAEs in face detection while maintaining generalization to generic object detection?
• RQ4 What performance-efficiency trade-off can be achieved by scaling a supernet trained with AutoFAE across different hardware constraints?
• RQ5 To what extent does AutoFAE generalize to other detectors and datasets beyond face detection?

Key findings

• ASFD-D6 achieves 96.7/96.2/92.1 AP on WIDER Face test, outperforming all prior competitors by a significant margin.
• ASFD-D0 achieves over 320 FPS inference speed on V100 GPU with only 3.1ms latency on VGA-resolution images, demonstrating high efficiency.
• AutoFAE improves AP by 0.5–1.0 points across RetinaNet, FCOS, and Faster R-CNN on COCO, proving strong generalization to generic object detection.
• The performance gap between COCO and WIDER Face is primarily due to smaller relative face scales (55% of faces <0.02 scale) and higher instance density (99% of images have >150 faces), which existing FAEs fail to handle.
• The discovered AutoFAE uses sparse, top-down connections and adaptive operations—dilated convolutions only in deeper layers—enabling better feature aggregation for small and occluded faces.
• The supernet-based model scaling yields 7 distinct detectors across different latency constraints, all achieving superior performance-efficiency trade-offs.