[Paper Review] OrientedFormer: An End-to-End Transformer-Based Oriented Object Detector in Remote Sensing Images

Jiaqi Zhao, Zeyu Ding+5 more|arXiv (Cornell University)|Sep 29, 2024

Remote-Sensing Image Classification4 citations

TL;DR

OrientedFormer proposes an end-to-end transformer-based detector for oriented object detection in remote sensing images, introducing Gaussian positional encoding, Wasserstein self-attention, and oriented cross-attention to address angle encoding, geometric relation modeling, and feature misalignment. It achieves state-of-the-art performance with +1.16 and +1.21 AP50 on DIOR-R and DOTA-v1.0, respectively, while reducing training from 3× to 1× epochs.

ABSTRACT

Oriented object detection in remote sensing images is a challenging task due to objects being distributed in multi-orientation. Recently, end-to-end transformer-based methods have achieved success by eliminating the need for post-processing operators compared to traditional CNN-based methods. However, directly extending transformers to oriented object detection presents three main issues: 1) objects rotate arbitrarily, necessitating the encoding of angles along with position and size; 2) the geometric relations of oriented objects are lacking in self-attention, due to the absence of interaction between content and positional queries; and 3) oriented objects cause misalignment, mainly between values and positional queries in cross-attention, making accurate classification and localization difficult. In this paper, we propose an end-to-end transformer-based oriented object detector, consisting of three dedicated modules to address these issues. First, Gaussian positional encoding is proposed to encode the angle, position, and size of oriented boxes using Gaussian distributions. Second, Wasserstein self-attention is proposed to introduce geometric relations and facilitate interaction between content and positional queries by utilizing Gaussian Wasserstein distance scores. Third, oriented cross-attention is proposed to align values and positional queries by rotating sampling points around the positional query according to their angles. Experiments on six datasets DIOR-R, a series of DOTA, HRSC2016 and ICDAR2015 show the effectiveness of our approach. Compared with previous end-to-end detectors, the OrientedFormer gains 1.16 and 1.21 AP$_{50}$ on DIOR-R and DOTA-v1.0 respectively, while reducing training epochs from 3$ imes$ to 1$ imes$. The codes are available at https://github.com/wokaikaixinxin/OrientedFormer.

Motivation & Objective

• To address the challenge of detecting arbitrarily oriented objects in remote sensing images, which are difficult to localize and classify due to multi-orientation, dense arrangements, and scale variations.
• To overcome three key limitations in extending transformers to oriented object detection: angle encoding, lack of geometric relation modeling in self-attention, and misalignment in cross-attention.
• To design an end-to-end framework that eliminates post-processing like NMS and enables one-to-one label assignment, improving training efficiency and detection accuracy.
• To achieve competitive performance with CNN-based detectors while reducing training epochs from 3× to 1×, enhancing practical deployment.

Proposed method

• Gaussian positional encoding is introduced to jointly represent the position, size, and orientation (angle) of oriented bounding boxes using 2D Gaussian distributions.
• Wasserstein self-attention incorporates geometric relations between content queries by computing Gaussian Wasserstein distance scores, enabling interaction between content and positional queries.
• Oriented cross-attention rotates sampling points around positional queries according to object angles, aligning values with positional queries to reduce misalignment in feature aggregation.
• The model uses a standard transformer encoder-decoder architecture with learnable queries, where positional queries represent object centers and sampling points are rotated for attention computation.
• The method integrates these components into a unified end-to-end framework, avoiding hand-designed post-processing such as NMS.
• Experiments are conducted on six datasets: DIOR-R, DOTA-v1.0, DOTA-v1.5, HRSC2016, and ICDAR2015, with ablation studies validating each component’s contribution.

(a)

Experimental results

Research questions

• RQ1 Can a transformer-based detector effectively encode object angles, position, and size simultaneously in remote sensing images?
• RQ2 Can geometric relations between object queries be modeled effectively within the self-attention mechanism to improve detection performance?
• RQ3 Can feature misalignment in cross-attention be mitigated by rotating sampling points according to object orientation?
• RQ4 Does the proposed end-to-end framework outperform previous detectors in accuracy and training efficiency?
• RQ5 Can the model generalize across diverse remote sensing datasets with varying conditions, such as poor lighting and extreme weather?

Key findings

• OrientedFormer achieves a 1.16 AP50 gain over previous end-to-end detectors on the DIOR-R dataset, demonstrating superior accuracy.
• On DOTA-v1.0, the model improves AP50 by 1.21 points compared to prior end-to-end methods, confirming strong generalization.
• The model reduces training epochs from 3× to 1× while maintaining or improving performance, significantly enhancing training efficiency.
• Ablation studies confirm that each component—Gaussian positional encoding, Wasserstein self-attention, and oriented cross-attention—contributes to performance gains.
• Visualization shows that learned positional queries are localized at object centers, and sampling points align effectively with oriented boxes, confirming the mechanism’s correctness.
• The model performs robustly on challenging cases such as small objects, high aspect ratios, and low-light conditions, though some suboptimal results remain for extremely small or background-similar objects.

(b)