[Paper Review] G-Rep: Gaussian Representation for Arbitrary-Oriented Object Detection

Liping Hou, Ke Lu+3 more|arXiv (Cornell University)|May 24, 2022

Advanced Neural Network Applications4 citations

TL;DR

This paper proposes G-Rep, a unified Gaussian representation for arbitrary-oriented object detection that models oriented bounding boxes (OBB), quadrilateral bounding boxes (QBB), and point sets (PointSet) as Gaussian distributions. By optimizing parameters via maximum likelihood estimation and using Gaussian metrics for both label assignment and regression loss, G-Rep achieves state-of-the-art performance across DOTA, HRSC2016, UCAS-AOD, and ICDAR2015, overcoming representation-specific issues like boundary discontinuity, representation ambiguity, and outlier sensitivity.

ABSTRACT

Typical representations for arbitrary-oriented object detection tasks include oriented bounding box (OBB), quadrilateral bounding box (QBB), and point set (PointSet). Each representation encounters problems that correspond to its characteristics, such as the boundary discontinuity, square-like problem, representation ambiguity, and isolated points, which lead to inaccurate detection. Although many effective strategies have been proposed for various representations, there is still no unified solution. Current detection methods based on Gaussian modeling have demonstrated the possibility of breaking this dilemma; however, they remain limited to OBB. To go further, in this paper, we propose a unified Gaussian representation called G-Rep to construct Gaussian distributions for OBB, QBB, and PointSet, which achieves a unified solution to various representations and problems. Specifically, PointSet or QBB-based object representations are converted into Gaussian distributions, and their parameters are optimized using the maximum likelihood estimation algorithm. Then, three optional Gaussian metrics are explored to optimize the regression loss of the detector because of their excellent parameter optimization mechanisms. Furthermore, we also use Gaussian metrics for sampling to align label assignment and regression loss. Experimental results on several public available datasets, such as DOTA, HRSC2016, UCAS-AOD, and ICDAR2015, show the excellent performance of the proposed method for arbitrary-oriented object detection.

Motivation & Objective

• To address inherent limitations in existing arbitrary-oriented object detection representations—such as boundary discontinuity in OBB, representation ambiguity in QBB, and outlier sensitivity in PointSet.
• To unify diverse object representations (OBB, QBB, PointSet) under a single differentiable framework that enables joint optimization.
• To develop a consistent label assignment and regression loss mechanism using Gaussian metrics, aligning training objectives with geometric similarity.
• To eliminate the need for task-specific loss designs or anchor presets by leveraging Gaussian distribution properties for end-to-end optimization.

Proposed method

• Convert PointSet or QBB representations into Gaussian distributions using a differentiable transformation, enabling parameter learning via maximum likelihood estimation (MLE).
• Design three Gaussian metrics (e.g., Bhattacharyya, Kullback-Leibler, and Mahalanobis) to compute similarity between predicted and ground-truth Gaussian distributions for regression loss.
• Use the same Gaussian metrics for label assignment to ensure consistency between positive sample selection and regression loss optimization.
• Optimize the detector using a unified loss function based on Gaussian metrics, replacing traditional IoU or L_n losses.
• Transform the final output from learned Gaussian distributions into OBB format for inference, preserving localization accuracy.
• Integrate the Gaussian representation into a standard detection head (e.g., RetinaNet or YOLO-based) without architectural overhaul.

Experimental results

Research questions

• RQ1 Can a unified Gaussian representation effectively model diverse arbitrary-oriented object representations (OBB, QBB, PointSet) while maintaining differentiability and optimization stability?
• RQ2 How do Gaussian metrics compare to traditional IoU or L_n losses in terms of robustness to outliers and boundary discontinuities in object detection?
• RQ3 Does aligning label assignment and regression loss using the same Gaussian metric improve detection performance across multiple datasets and object types?
• RQ4 To what extent does G-Rep outperform existing methods in handling representation-specific challenges like angle discontinuity, square-like problems, and point ordering ambiguity?
• RQ5 Can the proposed Gaussian-based framework generalize across diverse datasets, including aerial and text detection benchmarks?

Key findings

• G-Rep (PointSet) achieves 90.16% mAP on DOTA, outperforming state-of-the-art methods like RIDet and DAL, demonstrating superior generalization and robustness.
• On HRSC2016, G-Rep (PointSet) achieves 90.67% mAP, surpassing RIDet (OBB) and RIDet (QBB) by 0.93% and 0.54%, respectively, despite not using anchors or complex hyperparameter tuning.
• On UCAS-AOD, G-Rep (PointSet) achieves 89.64% mAP, showing consistent superiority over baseline models across diverse object shapes and orientations.
• On ICDAR2015 for oriented text detection, G-Rep (PointSet) achieves 81.3% F-measure, outperforming one-stage RO 3D and two-stage SCRDet in both recall and F-measure.
• Visualization results confirm that G-Rep points are concentrated inside objects, reducing boundary sensitivity and outlier impact compared to PointSet, which places points at object edges.
• The method effectively mitigates the square-like problem and boundary discontinuity in OBB by learning smooth, continuous Gaussian distributions instead of discrete angle-based boxes.