[Paper Review] DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution

Siyuan Qiao, Liang-Chieh Chen+1 more|arXiv (Cornell University)|Jun 3, 2020

Advanced Neural Network ApplicationsComputer Science88 references155 citations

TL;DR

DetectoRS combines Recursive Feature Pyramid and Switchable Atrous Convolution to enhance backbones, achieving state-of-the-art COCO performance for object detection, instance segmentation, and panoptic segmentation.

ABSTRACT

Many modern object detectors demonstrate outstanding performances by using the mechanism of looking and thinking twice. In this paper, we explore this mechanism in the backbone design for object detection. At the macro level, we propose Recursive Feature Pyramid, which incorporates extra feedback connections from Feature Pyramid Networks into the bottom-up backbone layers. At the micro level, we propose Switchable Atrous Convolution, which convolves the features with different atrous rates and gathers the results using switch functions. Combining them results in DetectoRS, which significantly improves the performances of object detection. On COCO test-dev, DetectoRS achieves state-of-the-art 55.7% box AP for object detection, 48.5% mask AP for instance segmentation, and 50.0% PQ for panoptic segmentation. The code is made publicly available.

Motivation & Objective

• Motivate and implement a backbone that looks and thinks twice for object detection.
• Develop macro-level recursion in feature pyramids to enrich multi-scale representations.
• Introduce micro-level Switchable Atrous Convolution to adapt receptive fields across scales.
• Demonstrate performance gains on COCO across detection, segmentation, and panoptic tasks.
• Provide a plug-and-play approach compatible with pretrained backbones.

Proposed method

• Propose Recursive Feature Pyramid (RFP) that adds feedback from top-down FPN layers into bottom-up backbone to create a multi-pass backbone representation.
• Introduce Switchable Atrous Convolution (SAC) that convolves features with multiple atrous rates and fuses outputs via location-dependent switches.
• Use ASPP as the connecting module to transform and fuse RFP features with the backbone.
• Apply a fusion module to update and blend features across unrolled RFP steps.
• Replace backbone 3x3 convolutions with SAC to enable easy conversion from pretrained networks with a weight-locking scheme.
• Experimentally integrate RFP and SAC into HTC to evaluate gains on COCO.

Experimental results

Research questions

• RQ1 How can feedback in the backbone (RFP) improve multi-scale representations for object detection?
• RQ2 Can Switchable Atrous Convolution (SAC) improve detection across object scales without retraining from scratch?
• RQ3 What are the combined benefits of macro-level (RFP) and micro-level (SAC) designs on standard detectors?
• RQ4 How do RFP and SAC affect bounding box detection, instance segmentation, and panoptic segmentation on COCO?

Key findings

• RFP improves box AP from 42.0 to 46.2 with ResNet-50 baseline.
• SAC improves box AP to 46.3 with ResNet-50, indicating strong micro-level gains.
• Combined DetectoRS (RFP + SAC) with ResNeXt-101-64x4d achieves 55.7% box AP and 48.5% mask AP on COCO test-dev.
• DetectoRS sets state-of-the-art 55.7% box AP and 48.5% mask AP (with ResNeXt-101-64x4d) and 50.0 PQ for panoptic segmentation on COCO test-dev.
• RFP + SAC also yields faster convergence and notable improvements in training losses compared to HTC baseline.
• Visualizations show learned switches align with object scales, and SAC increases receptive field for large objects.