[Paper Review] Learning RoI Transformer for Detecting Oriented Objects in Aerial Images

Jian Ding, Nan Xue+3 more|arXiv (Cornell University)|Dec 1, 2018

Robotics and Sensor-Based Localization3 references112 citations

TL;DR

The paper introduces RoI Transformer, a lightweight module that learns rotated RoIs from horizontal RoIs and extracts rotation-invariant features to detect oriented objects in aerial images, achieving state-of-the-art results with efficient inference.

ABSTRACT

Object detection in aerial images is an active yet challenging task in computer vision because of the birdview perspective, the highly complex backgrounds, and the variant appearances of objects. Especially when detecting densely packed objects in aerial images, methods relying on horizontal proposals for common object detection often introduce mismatches between the Region of Interests (RoIs) and objects. This leads to the common misalignment between the final object classification confidence and localization accuracy. Although rotated anchors have been used to tackle this problem, the design of them always multiplies the number of anchors and dramatically increases the computational complexity. In this paper, we propose a RoI Transformer to address these problems. More precisely, to improve the quality of region proposals, we first designed a Rotated RoI (RRoI) learner to transform a Horizontal Region of Interest (HRoI) into a Rotated Region of Interest (RRoI). Based on the RRoIs, we then proposed a Rotated Position Sensitive RoI Align (RPS-RoI-Align) module to extract rotation-invariant features from them for boosting subsequent classification and regression. Our RoI Transformer is with light weight and can be easily embedded into detectors for oriented object detection. A simple implementation of the RoI Transformer has achieved state-of-the-art performances on two common and challenging aerial datasets, i.e., DOTA and HRSC2016, with a neglectable reduction to detection speed. Our RoI Transformer exceeds the deformable Position Sensitive RoI pooling when oriented bounding-box annotations are available. Extensive experiments have also validated the flexibility and effectiveness of our RoI Transformer. The results demonstrate that it can be easily integrated with other detector architectures and significantly improve the performances.

Motivation & Objective

• Motivate accurate detection of oriented and densely packed objects in aerial images where horizontal RoIs cause misalignment.
• Propose a light, end-to-end trainable RoI Transformer to transform HRoIs into RRoIs and extract rotation-invariant features.
• Reduce computational complexity compared with extensive rotated-anchor approaches while improving accuracy.

Proposed method

• Introduce RRoI Learner to transform HRoIs into Rotated RoIs via a small fully connected regression head.
• Apply Rotated Position Sensitive RoI Align to pool rotation-invariant features from RRoIs.
• Use a light-head architecture to keep RoI-wise computations efficient.
• Train with IoU-based matching between RRoIs and RRoTs (rotated ground-truth) to improve supervision.
• Provide end-to-end differentiable RoI Transformer for integration with existing detectors.

Experimental results

Research questions

• RQ1 Can a learned transformation from horizontal to rotated RoIs improve alignment with oriented objects in aerial images?
• RQ2 Does Rotated PS RoI Align provide rotation-invariant features that boost classification and localization for oriented objects?
• RQ3 How does RoI Transformer compare to deformable RoI pooling and baseline light-head detectors in accuracy and efficiency on DOTA and HRSC2016?

Key findings

• RoI Transformer yields state-of-the-art or competitive mAP on DOTA and HRSC2016 datasets.
• Adding RoI Transformer to baseline Light-Head OBB improves mAP by up to 4.87 points in ablations.
• RoI Transformer demonstrates better handling of densely packed and long/thin objects, with significant gains over prior methods (e.g., ships in DOTA).
• Compared to deformable PS RoI pooling, RoI Transformer provides higher accuracy with lighter regression targets and rotation-aware alignment.
• Inference speed and memory remain favorable relative to competing rotated-ROI approaches (e.g., ~0.17s per image on 1024x1024 with TITAN X).
• RoI Transformer can be easily embedded into other detector architectures to improve oriented-object detection.