[Paper Review] Learning Modulated Loss for Rotated Object Detection

Wen Qian, Xue Yang+3 more|arXiv (Cornell University)|Nov 19, 2019

Advanced Image and Video Retrieval Techniques54 references63 citations

TL;DR

The paper identifies rotation sensitivity error in rotated object detectors and fixes it with a modulated rotation loss and eight-parameter regression, achieving state-of-the-art on DOTA and good generalization.

ABSTRACT

Popular rotated detection methods usually use five parameters (coordinates of the central point, width, height, and rotation angle) to describe the rotated bounding box and l1-loss as the loss function. In this paper, we argue that the aforementioned integration can cause training instability and performance degeneration, due to the loss discontinuity resulted from the inherent periodicity of angles and the associated sudden exchange of width and height. This problem is further pronounced given the regression inconsistency among five parameters with different measurement units. We refer to the above issues as rotation sensitivity error (RSE) and propose a modulated rotation loss to dismiss the loss discontinuity. Our new loss is combined with the eight-parameter regression to further solve the problem of inconsistent parameter regression. Experiments show the state-of-art performances of our method on the public aerial image benchmark DOTA and UCAS-AOD. Its generalization abilities are also verified on ICDAR2015, HRSC2016, and FDDB. Qualitative improvements can be seen in Fig 1, and the source code will be released with the publication of the paper.

Motivation & Objective

• Motivate improved stability and accuracy for rotated object detection by addressing loss discontinuity and parameter regression inconsistency.
• Formulate rotation sensitivity error (RSE) as rotation-boundary loss discontinuity and unit-inconsistency among parameters.
• Propose a modulated rotation loss for five-parameter models to smooth learning.
• Adopt eight-parameter regression to eliminate regression unit inconsistency and improve stability.
• Demonstrate state-of-the-art performance on DOTA and robust transfer to ICDAR2015, HRSC2016, and UCAS-AOD.

Proposed method

• Define five-parameter (OpenCV-style) and eight-parameter bounding box representations.
• Identify rotation sensitivity error (RSE) due to angular periodicity and width/height exchange in five-parameter models and regression unit mismatch.
• Introduce modulated rotation loss (ell_mr) that blends a standard L1-like term with a symmetry-based correction to smooth the loss near boundaries (Eq. 4).
• Express ell_mr for five-parameter regression using normalized coordinates and aspect ratio (Eq. 5-4).
• Address residual discontinuities in the eight-parameter model with ell_8p_mr that shifts vertex orderings and takes the minimum across three alignment scenarios (Eq. 6).
• Show that RSDet, built on RetinaNet-H with eight-parameter regression and ell_mr, trains stably and outperforms baselines on rotated object benchmarks.

Experimental results

Research questions

• RQ1 What causes training instability and performance drops in rotated object detectors when using standard five-parameter or eight-parameter representations?
• RQ2 Can a modulated loss resolve loss discontinuities caused by angle periodicity and width-height exchanges?
• RQ3 Does eight-parameter regression mitigate unit-inconsistency and improve regression stability for rotated boxes?
• RQ4 How does the proposed RSDet perform compared to state-of-the-art methods on DOTA and generalize to other datasets?

Key findings

• Five-parameter models with ell_mr improve mAP from 62.14 to 64.49 on DOTA with ResNet-50 RetinaNet-H.
• Eight-parameter regression with ell_mr further improves mAP to 66.77 on DOTA.
• RSDet with eight-parameter regression shows training stability improvements (Fig. 7) and higher accuracy across backbones (ResNet-50/101/152).
• On DOTA, RSDet with ResNet-152+Refine achieves 74.1 mAP in ablations; RSDet variants outperform several state-of-the-art methods (e.g., RRPN, SCRDet, R3 Det) in Table 3.
• ICDAR2015 and HRSC2016 generalize well, with ell_mr and eight-parameter regression yielding better APs over baselines (Table 5).
• UCAS-AOD results show RSDet achieving 96.50% AP for the Plane and Car tasks, outperforming existing methods (Table 6).