[Paper Review] Multi-Branch Fully Convolutional Network for Face Detection

Yancheng Bai, Bernard Ghanem|arXiv (Cornell University)|Jul 20, 2017

Face recognition and analysis27 references3 citations

TL;DR

This paper proposes a Multi-Branch Fully Convolutional Network (MB-FCN) for face detection that enables single-shot inference across all face scales by leveraging scale-specific skip connections from multiple convolutional layers. The method achieves state-of-the-art performance on FDDB and WIDER FACE, especially on the challenging Hard subset, while running at 15 FPS on 640×480 images without assuming a minimum detectable face size.

ABSTRACT

Face detection is a fundamental problem in computer vision. It is still a challenging task in unconstrained conditions due to significant variations in scale, pose, expressions, and occlusion. In this paper, we propose a multi-branch fully convolutional network (MB-FCN) for face detection, which considers both efficiency and effectiveness in the design process. Our MB-FCN detector can deal with faces at all scale ranges with only a single pass through the backbone network. As such, our MB-FCN model saves computation and thus is more efficient, compared to previous methods that make multiple passes. For each branch, the specific skip connections of the convolutional feature maps at different layers are exploited to represent faces in specific scale ranges. Specifically, small faces can be represented with both shallow fine-grained and deep powerful coarse features. With this representation, superior improvement in performance is registered for the task of detecting small faces. We test our MB-FCN detector on two public face detection benchmarks, including FDDB and WIDER FACE. Extensive experiments show that our detector outperforms state-of-the-art methods on all these datasets in general and by a substantial margin on the most challenging among them (e.g. WIDER FACE Hard subset). Also, MB-FCN runs at 15 FPS on a GPU for images of size 640 x 480 with no assumption on the minimum detectable face size.

Motivation & Objective

• To address the challenge of detecting faces under extreme variations in scale, pose, expression, and occlusion.
• To improve detection accuracy for small faces by combining shallow fine-grained and deep coarse features.
• To enhance computational efficiency by eliminating the need for multi-scale image pyramids.
• To achieve high performance across all scale ranges with a single forward pass through the network.
• To eliminate assumptions on minimum detectable face size while maintaining real-time inference speed.

Proposed method

• The MB-FCN architecture uses multiple branches, each dedicated to detecting faces within specific scale ranges.
• Each branch employs skip connections from different convolutional layers to fuse features across spatial and semantic levels.
• Shallow features (high-resolution) and deep features (semantic-rich) are combined to represent small faces more effectively.
• The network uses fully convolutional heads for per-location classification and bounding box regression.
• Non-maximum suppression is applied across all branches to merge detections and produce final results.
• The model is trained end-to-end on WIDER FACE and evaluated on both WIDER FACE and FDDB benchmarks.

Experimental results

Research questions

• RQ1 Can a single-shot, multi-branch FCN architecture outperform multi-scale pyramid methods in face detection accuracy and efficiency?
• RQ2 How does combining shallow and deep features improve detection of small faces?
• RQ3 Can scale-specific skip connections enhance performance on challenging subsets like WIDER FACE Hard?
• RQ4 What is the trade-off between accuracy and inference speed when eliminating image pyramids?
• RQ5 To what extent does the absence of a minimum face size assumption affect detection performance?

Key findings

• The MB-FCN detector achieves state-of-the-art performance on both FDDB and WIDER FACE, with a 14.0% absolute improvement on the WIDER FACE Hard subset compared to CMS-RCNN.
• On the WIDER FACE Hard subset, MB-FCN outperforms all prior methods, including those fine-tuned on FDDB, despite being trained only on WIDER FACE.
• The model runs at 15 FPS on a GPU for 640×480 images, with no assumption on the minimum detectable face size.
• The use of scale-specific skip connections significantly improves small face detection, as evidenced by superior performance on small-scale benchmarks.
• The multi-scale inference strategy (MB-FCN-ms) achieves better accuracy (+4.0% on Hard subset) but runs five times slower, confirming the efficiency of the single-pass design.
• Qualitative results show robust detection of faces with extreme poses, occlusion, and expressions, though failures occur on severely truncated or low-resolution faces.