[Paper Review] Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation

Zhaohui Zheng, Ping Wang+5 more|arXiv (Cornell University)|May 7, 2020

Advanced Neural Network Applications55 references83 citations

TL;DR

The paper introduces Complete-IoU (CIoU) loss and Cluster-NMS to incorporate three geometric factors into bounding box regression and NMS, improving AP and AR with real-time inference across detection and segmentation models.

ABSTRACT

Deep learning-based object detection and instance segmentation have achieved unprecedented progress. In this paper, we propose Complete-IoU (CIoU) loss and Cluster-NMS for enhancing geometric factors in both bounding box regression and Non-Maximum Suppression (NMS), leading to notable gains of average precision (AP) and average recall (AR), without the sacrifice of inference efficiency. In particular, we consider three geometric factors, i.e., overlap area, normalized central point distance and aspect ratio, which are crucial for measuring bounding box regression in object detection and instance segmentation. The three geometric factors are then incorporated into CIoU loss for better distinguishing difficult regression cases. The training of deep models using CIoU loss results in consistent AP and AR improvements in comparison to widely adopted $\ell_n$-norm loss and IoU-based loss. Furthermore, we propose Cluster-NMS, where NMS during inference is done by implicitly clustering detected boxes and usually requires less iterations. Cluster-NMS is very efficient due to its pure GPU implementation, and geometric factors can be incorporated to improve both AP and AR. In the experiments, CIoU loss and Cluster-NMS have been applied to state-of-the-art instance segmentation (e.g., YOLACT and BlendMask-RT), and object detection (e.g., YOLO v3, SSD and Faster R-CNN) models. Taking YOLACT on MS COCO as an example, our method achieves performance gains as +1.7 AP and +6.2 AR$_{100}$ for object detection, and +0.9 AP and +3.5 AR$_{100}$ for instance segmentation, with 27.1 FPS on one NVIDIA GTX 1080Ti GPU. All the source code and trained models are available at https://github.com/Zzh-tju/CIoU

Motivation & Objective

• Motivate bounding box regression limitations in IoU-based losses for detection and segmentation.
• Propose a complete geometric-factor loss (CIoU) that includes overlap, center distance, and aspect-ratio terms.
• Develop Cluster-NMS to accelerate non-maximum suppression while enabling geometric-factor integration.
• Demonstrate training and inference gains on state-of-the-art detectors and segmenters.
• Show real-time performance on GPUs without sacrificing accuracy.

Proposed method

• Define CIoU loss as 1 - IoU plus normalized center distance and an aspect-ratio term with an adaptive weight (alpha).
• Formulate three geometric factors S (overlap), D (distance), V (aspect ratio) as scale-invariant and normalized to [0,1].
• Provide analysis and simulation to compare CIoU against IoU and GIoU losses, showing faster convergence and better regression for extreme cases.
• Introduce Cluster-NMS that groups boxes into clusters and performs NMS on GPUs with less iterations.
• Incorporate geometric factors into Cluster-NMS via score penalties and distance-based terms (Cluster-NMS_S, Cluster-NMS_S+D, Cluster-NMS_W, Cluster-NMS_W+D).
• Apply CIoU and Cluster-NMS to YOLACT, BlendMask-RT, YOLOv3, SSD, and Faster R-CNN to validate gains.

Experimental results

Research questions

• RQ1 Do CIoU loss and Cluster-NMS improve bounding box regression and suppression quality compared to traditional losses and NMS variants?
• RQ2 How do the three geometric factors (overlap area, center distance, aspect ratio) influence training dynamics and convergence?
• RQ3 Can CIoU and Cluster-NMS maintain or improve inference speed when integrated into state-of-the-art detectors and segmenters?
• RQ4 Are the methods effective across both object detection and instance segmentation tasks?

Key findings

• CIoU loss yields consistent AP and AR improvements over l1-norm and IoU-based losses.
• Cluster-NMS provides significant AP and AR gains while maintaining real-time inference.
• Applied to YOLACT on MS COCO, the method achieves +1.7 AP and +6.2 AR100 for object detection and +1.1 AP and +3.5 AR100 for instance segmentation, with 27.1 FPS on a GTX 1080Ti.
• Applied to other models (YOLOv3, SSD, Faster R-CNN) with observed gains.
• CIoU converges faster than GIoU and handles extreme aspect ratios better.
• Cluster-NMS can be implemented purely on GPU and can replicate original NMS results with fewer iterations.