[Paper Review] Oriented R-CNN for Object Detection

Xingxing Xie, Gong Cheng+3 more|arXiv (Cornell University)|Aug 12, 2021

Advanced Neural Network Applications43 references4 citations

TL;DR

This paper proposes Oriented R-CNN, a two-stage object detection framework that uses a lightweight oriented Region Proposal Network (oriented RPN) to generate high-quality oriented proposals with minimal computational cost. By introducing a midpoint offset representation that extends standard RPN regression to six parameters, the method achieves state-of-the-art mAP of 75.87% on DOTA and 96.50% on HRSC2016 at 15.1 FPS with a single RTX 2080Ti, outperforming both one-stage and two-stage detectors in accuracy-efficiency trade-off.

ABSTRACT

Current state-of-the-art two-stage detectors generate oriented proposals through time-consuming schemes. This diminishes the detectors' speed, thereby becoming the computational bottleneck in advanced oriented object detection systems. This work proposes an effective and simple oriented object detection framework, termed Oriented R-CNN, which is a general two-stage oriented detector with promising accuracy and efficiency. To be specific, in the first stage, we propose an oriented Region Proposal Network (oriented RPN) that directly generates high-quality oriented proposals in a nearly cost-free manner. The second stage is oriented R-CNN head for refining oriented Regions of Interest (oriented RoIs) and recognizing them. Without tricks, oriented R-CNN with ResNet50 achieves state-of-the-art detection accuracy on two commonly-used datasets for oriented object detection including DOTA (75.87% mAP) and HRSC2016 (96.50% mAP), while having a speed of 15.1 FPS with the image size of 1024$ imes$1024 on a single RTX 2080Ti. We hope our work could inspire rethinking the design of oriented detectors and serve as a baseline for oriented object detection. Code is available at https://github.com/jbwang1997/OBBDetection.

Motivation & Objective

• To address the computational bottleneck in state-of-the-art two-stage oriented object detectors caused by expensive oriented proposal generation.
• To design a simple, efficient, and accurate two-stage framework for arbitrary-oriented object detection.
• To reduce the parameter count and inference cost of proposal generation while maintaining high recall and localization accuracy.
• To establish a new baseline for oriented object detection that balances accuracy and speed.
• To inspire rethinking of oriented detector design and regression schemes through a novel representation method.

Proposed method

• Proposes a lightweight oriented RPN that extends standard RPN by increasing the regression output from four to six parameters to predict oriented bounding boxes.
• Introduces the midpoint offset representation, which encodes oriented objects using six parameters (midpoint coordinates, width, height, and angle), enabling bounded and stable regression.
• Uses rotated RoI alignment in the second stage to extract features from oriented proposals and perform classification and regression.
• Employs a fully convolutional architecture for the oriented RPN, minimizing parameters and reducing overfitting risk.
• Adopts ResNet-50-FPN as the backbone network and uses multi-scale training and testing to improve performance.
• Selects top 1000 proposals per image at inference time to balance speed and accuracy.

Experimental results

Research questions

• RQ1 Can a simple, lightweight oriented RPN be designed to generate high-quality oriented proposals without incurring significant computational cost?
• RQ2 How does the midpoint offset representation improve the stability and efficiency of oriented object regression compared to anchor-based or transformer-based methods?
• RQ3 Can a two-stage detector achieve state-of-the-art accuracy while maintaining inference speeds comparable to one-stage detectors?
• RQ4 What is the impact of proposal quality (e.g., recall at top-N) on final detection mAP in oriented object detection?
• RQ5 How does the proposed framework compare in accuracy and speed to existing one-stage and two-stage oriented detectors on standard benchmarks?

Key findings

• Oriented R-CNN achieves 75.87% mAP on the DOTA dataset using ResNet-50-FPN, surpassing all existing state-of-the-art methods, including those with deeper backbones.
• On the HRSC2016 dataset, the method achieves 96.50% mAP with ResNet-50-FPN, setting a new state-of-the-art accuracy.
• With multi-scale training and testing, the mAP on DOTA increases to 80.87% using the same ResNet-50-FPN backbone.
• The model runs at 15.1 FPS on a single RTX 2080Ti with 1024×1024 input resolution, demonstrating high efficiency.
• The oriented RPN achieves 92.80% recall with 2000 proposals and only a 0.6% drop when reduced to 1000 proposals, indicating strong proposal quality.
• The oriented RPN has only about 1/3000 the parameters of RoI Transformer+ and 1/15 of rotated RPN, significantly reducing model complexity.