[Paper Review] RepPoints V2: Verification Meets Regression for Object Detection

Yihong Chen, Zheng Zhang+4 more|arXiv (Cornell University)|Jul 16, 2020

Advanced Neural Network Applications32 references70 citations

TL;DR

RepPoints v2 integrates verification tasks as auxiliary branches into a regression-based detector to improve localization and detection accuracy, achieving about 2.0 mAP gains on COCO and 52.1 mAP test-dev with a single model.

ABSTRACT

Verification and regression are two general methodologies for prediction in neural networks. Each has its own strengths: verification can be easier to infer accurately, and regression is more efficient and applicable to continuous target variables. Hence, it is often beneficial to carefully combine them to take advantage of their benefits. In this paper, we take this philosophy to improve state-of-the-art object detection, specifically by RepPoints. Though RepPoints provides high performance, we find that its heavy reliance on regression for object localization leaves room for improvement. We introduce verification tasks into the localization prediction of RepPoints, producing RepPoints v2, which provides consistent improvements of about 2.0 mAP over the original RepPoints on the COCO object detection benchmark using different backbones and training methods. RepPoints v2 also achieves 52.1 mAP on COCO exttt{test-dev} by a single model. Moreover, we show that the proposed approach can more generally elevate other object detection frameworks as well as applications such as instance segmentation. The code is available at https://github.com/Scalsol/RepPointsV2.

Motivation & Objective

• Explore whether verification can improve regression-based object detectors without disrupting the detector’s flow.
• Identify effective verification tasks that complement regression-based localization (e.g., corner verification, within-box foreground).
• Propose a general fusion framework to integrate verification cues as auxiliary branches.
• Demonstrate RepPoints v2 improvements across backbones and training regimes on COCO.
• Show generalizability to other detectors and tasks like instance segmentation.

Proposed method

• Introduce auxiliary verification branches (corner verification and within-box foreground verification) as side-branches to the main regression head.
• Use explicit-corners variant in RepPoints to define bounding boxes via two corner points, enabling compatible fusion with verification cues.
• Fuse verification outputs with the main feature map via a 1x1 embedding and elementwise addition, enabling multi-task learning and feature enhancement.
• Apply joint inference that refines corner positions by leveraging verification scores alongside regression predictions.
• Train with a multi-task loss L = L_RepPoints + λ1 L_corner + λ2 L_foreground, with λ1=0.25 and λ2=0.1.
• Extend the fusion methodology to other detectors such as FCOS and to instance segmentation (Dense RepPoints) to demonstrate generality.

Experimental results

Research questions

• RQ1 Can verification tasks be effectively integrated with a regression-based detector without disrupting intermediate representations?
• RQ2 What forms of verification (corner-based, within-box foreground) best improve localization and detection performance?
• RQ3 How does fusion (multi-task learning, feature enhancement, joint inference) contribute to overall gains?
• RQ4 Is RepPoints v2 transferable to other detectors and tasks like instance segmentation?

Key findings

• RepPoints v2 achieves about 2.0 mAP improvement over the original RepPoints on COCO with different backbones and training methods.
• RepPoints v2 reaches 52.1 mAP on COCO test-dev with a single ResNeXt-101-DCN model (with multi-scale testing).
• Corner verification alone provides ≈1.4 mAP gains, mainly boosting higher IoU (AP90) while AP50 changes little; foreground verification adds ≈0.5 mAP mainly at lower IoU (AP50).
• The full RepPoints v2 fusion (multi-task, feature enhancement, and joint inference) yields an overall ~1.9 mAP improvement over the baseline without verification modules.
• Applying the approach to FCOS yields a 1.3 mAP gain on COCO test-dev with ResNet-50; Dense RepPoints with verification improves Dense mask AP by 1.3 on COCO test-dev.