[Paper Review] Light-Head R-CNN: In Defense of Two-Stage Object Detector

Zeming Li, Chao Peng+4 more|arXiv (Cornell University)|Nov 20, 2017

Advanced Neural Network ApplicationsComputer Science1 references284 citations

TL;DR

The paper proposes a light-head two-stage detector that uses thin feature maps and a cheap R-CNN subnet to achieve high accuracy with high speed, outperforming many single-stage and two-stage detectors on COCO.

ABSTRACT

In this paper, we first investigate why typical two-stage methods are not as fast as single-stage, fast detectors like YOLO and SSD. We find that Faster R-CNN and R-FCN perform an intensive computation after or before RoI warping. Faster R-CNN involves two fully connected layers for RoI recognition, while R-FCN produces a large score maps. Thus, the speed of these networks is slow due to the heavy-head design in the architecture. Even if we significantly reduce the base model, the computation cost cannot be largely decreased accordingly. We propose a new two-stage detector, Light-Head R-CNN, to address the shortcoming in current two-stage approaches. In our design, we make the head of network as light as possible, by using a thin feature map and a cheap R-CNN subnet (pooling and single fully-connected layer). Our ResNet-101 based light-head R-CNN outperforms state-of-art object detectors on COCO while keeping time efficiency. More importantly, simply replacing the backbone with a tiny network (e.g, Xception), our Light-Head R-CNN gets 30.7 mmAP at 102 FPS on COCO, significantly outperforming the single-stage, fast detectors like YOLO and SSD on both speed and accuracy. Code will be made publicly available.

Motivation & Objective

• Motivate why conventional two-stage detectors lag in speed compared to single-stage detectors.
• Design a lighter head for two-stage detectors to improve speed without sacrificing accuracy.
• Demonstrate that thin feature maps and a simple R-CNN subnetwork can boost efficiency and performance.
• Showcase the flexibility to use small backbones while maintaining competitive accuracy on COCO.

Proposed method

• Introduce a light-head two-stage detector architecture with a thin feature map produced by large-kernel separable convolution.
• Attach a cheap R-CNN subnetwork consisting of a single fully-connected layer (2048 channels) for RoI classification and regression.
• Use thin feature maps before RoI pooling to reduce computation and memory, enabling efficient RoI warping.
• Employ large-kernel separable convolutions to enrich thin feature maps (k=15, C_mid=256/64, C_out=490) before RoI pooling.
• Apply RoI pooling (PSRoI/ROI pooling) on the thin maps followed by the lightweight R-CNN subnet.
• Experiment with setting L (ResNet-101 backbone) and setting S (tiny Xception-like backbone) to demonstrate speed-accuracy trade-offs.

Experimental results

Research questions

• RQ1 Can a lighter head in two-stage detectors close the speed gap with single-stage detectors without sacrificing accuracy?
• RQ2 Do thin feature maps and a simple RoI subnetwork provide competitive performance across backbones—from large to tiny?

Key findings

• With a ResNet-101 backbone, Light-Head R-CNN achieves 37.7 mmAP when combined with large-kernel feature maps and the light R-CNN head.
• On COCO test-dev (single-scale), Light-Head R-CNN reaches 39.5–41.5 mmAP depending on refinements, outperforming several fast detectors.
• Using a tiny Xception-like backbone, Light-Head R-CNN achieves 30.7 mmAP at 102 FPS, surpassing YOLO/SSD baselines in both speed and accuracy.
• Reducing RoI feature map channels to 490 (10×7×7) while keeping a 2048-channel FC head yields competitive mmAP (~31.4–35.2) with substantial speedups.
• Incorporating large kernel (k=15) in thin feature maps improves mmAP by ~0.7 points over the baseline thin-map variant.
• Adding multi-scale training and alignment-based pooling (PSRoI/ROI Align) provides additional mmAP gains (up to ~1.3 points from PSAlign and multi-scale).