[Paper Review] Receptive Field Block Net for Accurate and Fast Object Detection

Songtao Liu, Di Huang+1 more|arXiv (Cornell University)|Nov 21, 2017

Advanced Neural Network ApplicationsComputer Science39 references120 citations

TL;DR

The paper introduces Receptive Field Block (RFB) to enhance lightweight SSD-based detectors, achieving high accuracy comparable to deep backbones while preserving real-time speed. RFB Net outperforms several real-time detectors on VOC and COCO.

ABSTRACT

Current top-performing object detectors depend on deep CNN backbones, such as ResNet-101 and Inception, benefiting from their powerful feature representations but suffering from high computational costs. Conversely, some lightweight model based detectors fulfil real time processing, while their accuracies are often criticized. In this paper, we explore an alternative to build a fast and accurate detector by strengthening lightweight features using a hand-crafted mechanism. Inspired by the structure of Receptive Fields (RFs) in human visual systems, we propose a novel RF Block (RFB) module, which takes the relationship between the size and eccentricity of RFs into account, to enhance the feature discriminability and robustness. We further assemble RFB to the top of SSD, constructing the RFB Net detector. To evaluate its effectiveness, experiments are conducted on two major benchmarks and the results show that RFB Net is able to reach the performance of advanced very deep detectors while keeping the real-time speed. Code is available at https://github.com/ruinmessi/RFBNet.

Motivation & Objective

• Motivate real-time object detection with high accuracy without resorting to very deep backbones.
• Leverage neuroscience-inspired receptive field characteristics to improve feature discriminability in lightweight networks.
• Develop a practical detector by inserting RFB modules atop SSD and evaluate on VOC and COCO.

Proposed method

• Propose Receptive Field Block (RFB) comprising multi-branch convolutions with varying kernels to simulate multiple RF sizes.
• Use bottleneck structures and factorized convolutions (1x1, 3x3, and 1xN / Nx1 variants) with ResNet/Inception-style residual connections.
• Attach dilated pooling/convolution layers to model eccentricity-related RF expansion, and concatenate branches to form enhanced features.
• Replace top SSD layers with RFB (and an RFB-s variant for smaller RFs) to form RFB Net on a VGG16 backbone.
• Train with SSD-style losses, data augmentation, and hard negative mining; adopt a warmup learning rate strategy and standard schedules.
• For COCO, explore small architectural tweaks (up-sampling conv7, 7x7 branch) to boost accuracy with modest speed impact.

Experimental results

Research questions

• RQ1 Can an RF-inspired module on a lightweight backbone achieve competitive accuracy with real-time speeds?
• RQ2 Does simulating size-eccentricity relationships of receptive fields improve detection robustness and discriminability?
• RQ3 How does RFB Net compare to state-of-the-art one-stage and two-stage detectors on VOC and COCO?

Key findings

• RFB Net300 (VGG backbone) achieves 80.5% mAP on VOC2007 with real-time speed (83 FPS).
• RFB Net512 (VGG) reaches 82.2% mAP on VOC2007 with 38 FPS, outperforming many real-time and even several two-stage detectors.
• On COCO, RFB Net300 achieves 30.3% mAP on test-dev and 49.3% in the 0.5:0.95 range, surpassing SSD300* and approaching R-FCN-level performance with lighter backbones.
• RFB modules outperform Inception, ASPP, and Deformable CNN blocks when mounted on SSD for VOC2007 and COCO minival.
• COCO with RFB Net512-E (enhanced RFB) yields 34.4% mAP with modest speed increase, highlighting favorable speed-accuracy trade-offs.
• Linking RFB to MobileNet-SSD confirms generalization and improved accuracy with a lightweight backbone.