[Paper Review] CenterNet: Keypoint Triplets for Object Detection

Kaiwen Duan, Song Bai+4 more|arXiv (Cornell University)|Apr 17, 2019

Advanced Neural Network ApplicationsComputer Science45 references164 citations

TL;DR

CenterNet detects each object as a triplet of keypoints (center, top-left, bottom-right) and uses center pooling and cascade corner pooling to reduce false positives, achieving state-of-the-art one-stage AP on COCO (47.0) and competitive two-stage results.

ABSTRACT

In object detection, keypoint-based approaches often suffer a large number of incorrect object bounding boxes, arguably due to the lack of an additional look into the cropped regions. This paper presents an efficient solution which explores the visual patterns within each cropped region with minimal costs. We build our framework upon a representative one-stage keypoint-based detector named CornerNet. Our approach, named CenterNet, detects each object as a triplet, rather than a pair, of keypoints, which improves both precision and recall. Accordingly, we design two customized modules named cascade corner pooling and center pooling, which play the roles of enriching information collected by both top-left and bottom-right corners and providing more recognizable information at the central regions, respectively. On the MS-COCO dataset, CenterNet achieves an AP of 47.0%, which outperforms all existing one-stage detectors by at least 4.9%. Meanwhile, with a faster inference speed, CenterNet demonstrates quite comparable performance to the top-ranked two-stage detectors. Code is available at https://github.com/Duankaiwen/CenterNet.

Motivation & Objective

• Motivate improving one-stage keypoint-based object detectors by leveraging internal object region patterns.
• Extend CornerNet by introducing a center keypoint to form a keypoint triplet for robust object representation.
• Enrich corner and center features via center pooling and cascade corner pooling to improve precision and recall.
• Evaluate CenterNet on MS-COCO to quantify gains in AP and AR across object scales and compare with state-of-the-art detectors.

Proposed method

• Represent each object as a triplet consisting of a center keypoint and a pair of corners.
• Predict a center heatmap and corner heatmaps with embeddings and offsets as in CornerNet, then form bounding boxes from corner pairs.
• Define a scale-aware central region for each proposed bounding box and require a center keypoint of the same class within this region to validate the box.
• Introduce center pooling to enhance center keypoints by aggregating max responses in horizontal and vertical directions.
• Introduce cascade corner pooling to enrich corner features by combining boundary and internal directional max responses.
• Train with a multi-term loss including focal losses for corners and centers, pull/push embedding losses, and offset losses; perform inference with center verification and non-maximum suppression.

Experimental results

Research questions

• RQ1 Can a center keypoint within a central region improve the correctness of corner-based object detections?
• RQ2 Does enriching center and corner information with center pooling and cascade corner pooling improve AP and AR on COCO?
• RQ3 How does CenterNet perform compared to CornerNet and other state-of-the-art detectors on MS-COCO?
• RQ4 What is the impact of scale-aware central regions on detection of small versus large objects?

Key findings

• CenterNet achieves 47.0% AP on COCO test-dev with CenterNet102/104 backbones and multi-scale testing, outperforming all existing one-stage detectors by at least 4.9% AP.
• CenterNet reduces incorrect bounding boxes (false discovery) especially for small objects, compared to CornerNet.
• CenterNet with center pooling and cascade corner pooling shows gains in AP and AR over the CornerNet baseline, with larger gains for small and large objects depending on backbone and scale.
• The scale-aware central region improves recall for small boxes while maintaining precision for large boxes.
• CenterNet's single-scale AP with Hourglass-104 backbone reaches 44.9% (single-scale) and 47.4% (multi-scale), while multi-scale CenterNet with Hourglass-104 reaches 47.0% AP, competitive with top two-stage detectors.
• Inference speed remains practical (270–340 ms per image) while delivering significant accuracy improvements over the baseline.