[Paper Review] R3Det: Refined Single-Stage Detector with Feature Refinement for Rotating Object

Xue Yang, Junchi Yan+2 more|arXiv (Cornell University)|Aug 15, 2019

Advanced Image and Video Retrieval TechniquesComputer Science71 references160 citations

TL;DR

R3Det proposes a refined single-stage rotating object detector with a feature refinement module and an approximate SkewIoU loss to achieve accurate rotation estimation and high speed on aerial/text/remote sensing datasets.

ABSTRACT

Rotation detection is a challenging task due to the difficulties of locating the multi-angle objects and separating them effectively from the background. Though considerable progress has been made, for practical settings, there still exist challenges for rotating objects with large aspect ratio, dense distribution and category extremely imbalance. In this paper, we propose an end-to-end refined single-stage rotation detector for fast and accurate object detection by using a progressive regression approach from coarse to fine granularity. Considering the shortcoming of feature misalignment in existing refined single-stage detector, we design a feature refinement module to improve detection performance by getting more accurate features. The key idea of feature refinement module is to re-encode the position information of the current refined bounding box to the corresponding feature points through pixel-wise feature interpolation to realize feature reconstruction and alignment. For more accurate rotation estimation, an approximate SkewIoU loss is proposed to solve the problem that the calculation of SkewIoU is not derivable. Experiments on three popular remote sensing public datasets DOTA, HRSC2016, UCAS-AOD as well as one scene text dataset ICDAR2015 show the effectiveness of our approach. Tensorflow and Pytorch version codes are available at https://github.com/Thinklab-SJTU/R3Det_Tensorflow and https://github.com/SJTU-Thinklab-Det/r3det-on-mmdetection, and R3Det is also integrated in our open source rotation detection benchmark: https://github.com/yangxue0827/RotationDetection.

Motivation & Objective

• Motivate accurate rotation detection for objects with large aspect ratios, dense distributions, and arbitrary orientations.
• Develop an end-to-end refined single-stage detector that preserves speed while improving accuracy.
• Address feature misalignment during refinement with a dedicated feature refinement module (FRM).
• Introduce a differentiable, approximate SkewIoU loss to improve rotation estimation.
• Demonstrate state-of-the-art performance on multiple public rotating datasets without heavy multi-stage backbones.

Proposed method

• Adopt a progressive coarse-to-fine regression strategy that starts with horizontal anchors in early stages and refines with rotating anchors in later stages.
• Introduce a feature refinement module (FRM) that re-encodes refined bounding box positions into feature points via pixel-wise bilinear interpolation to reconstruct and align the feature map.
• Use a multi-task loss that combines a derivable regressor term with an approximate SkewIoU loss to better align gradients with rotation IoU.
• Implement bounding-box filtering (BF) and a large-kernel (LK) feature refinement step to efficiently reconstruct features for refined boxes.
• Apply multiple refinement stages with staged IoU thresholds to progressively improve localization while maintaining speed.

Experimental results

Research questions

• RQ1 How can a single-stage detector be made accurate for arbitrarily rotated objects without sacrificing speed?
• RQ2 Does a feature refinement module that reconstructs aligned features improve detection performance for rotated objects?
• RQ3 Can an approximate, differentiable SkewIoU loss provide better rotation regression than standard losses in rotation detection?
• RQ4 What is the impact of progressive coarse-to-fine regression with rotating anchors on dense, large-aspect-ratio object detection?
• RQ5 How many refinement stages are beneficial before diminishing returns set in rotation detection?

Key findings

• The proposed FRM-driven feature reconstruction yields measurable gains, improving mAP by about 0.8–2.8 points depending on configuration.
• Feature refinement with FRM provides substantial gains, with an ablation showing up to 2.79% improvement on DOTA when FRM is used.
• A progressive coarse-to-fine refinement strategy with multiple stages yields significant gains, but three or more refinements do not always increase overall performance beyond ensemble gains.
• An approximate SkewIoU loss function, particularly the exp-based variant, improves training stability and final rotation accuracy, outperforming PIoU baselines in reported experiments.
• On the DOTA dataset, the R3Det family achieves competitive to state-of-the-art results among single-stage methods and shows strong performance without multi-scale training/testing, with reported mAP values exceeding prior single-stage baselines.
• Across HRSC2016, UCAS-AOD, and ICDAR2015 datasets, R3Det and FRM consistently improve detection accuracy when compared to baselines using horizontal or rotating anchors.