[Paper Review] Bottom-up Object Detection by Grouping Extreme and Center Points

Xingyi Zhou, Jiacheng Zhuo+1 more|arXiv (Cornell University)|Jan 23, 2019

Advanced Neural Network ApplicationsComputer Science41 references107 citations

TL;DR

ExtremeNet detects four extreme points and a center point for each object category, then groups them geometrically to form bounding boxes, achieving competitive COCO results without region proposals.

ABSTRACT

With the advent of deep learning, object detection drifted from a bottom-up to a top-down recognition problem. State of the art algorithms enumerate a near-exhaustive list of object locations and classify each into: object or not. In this paper, we show that bottom-up approaches still perform competitively. We detect four extreme points (top-most, left-most, bottom-most, right-most) and one center point of objects using a standard keypoint estimation network. We group the five keypoints into a bounding box if they are geometrically aligned. Object detection is then a purely appearance-based keypoint estimation problem, without region classification or implicit feature learning. The proposed method performs on-par with the state-of-the-art region based detection methods, with a bounding box AP of 43.2% on COCO test-dev. In addition, our estimated extreme points directly span a coarse octagonal mask, with a COCO Mask AP of 18.9%, much better than the Mask AP of vanilla bounding boxes. Extreme point guided segmentation further improves this to 34.6% Mask AP.

Motivation & Objective

• Motivate and demonstrate that bottom-up object detection using extreme and center keypoints can match top-down region-based methods.
• Propose a purely appearance-based geometric grouping method to form object bounding boxes from predicted keypoints.
• Show that extreme points provide richer object information and enable coarse mask estimation through octagon fitting and optional DEXTR-based refinement.
• Evaluate ExtremeNet on COCO to compare with contemporary one-stage and two-stage detectors across bounding box and instance segmentation metrics.

Proposed method

• Predict five heatmaps per object category (four extreme points: top, left, bottom, right; one center point) using a state-of-the-art keypoint estimation network (HourglassNet).
• Group extreme points into detections by enumerating feasible quadruples and selecting those whose geometric center aligns with a center heatmap output.
• Use center grouping with thresholds to produce bounding boxes and detect confidence via the average score of the five keypoint heatmaps.
• Optionally apply edge aggregation to strengthen nearby edge responses and ghost box suppression to mitigate false positives.
• Approximate instance masks by fitting an octagon from the predicted extreme points and optionally refine with the DEXTR segmentation network using extreme points as guidance.

Experimental results

Research questions

• RQ1 Can bottom-up detection using four extreme points and a center point achieve competitive bounding-box AP compared to region-based detectors?
• RQ2 Does purely geometric grouping of extreme points, without associative embeddings or region proposals, reliably associate points to objects across varied scales and occlusions?
• RQ3 How well do extreme-point-based masks (octagons) and DEXTR-based refinement perform for instance segmentation relative to fully supervised methods?

Key findings

• ExtremeNet achieves 43.7% bounding-box AP on COCO test-dev with multi-scale testing, on par with state-of-the-art two-stage detectors and better than many one-stage methods.
• Center grouping with geometric reasoning provides effective object associations, with ablations showing significant drops when removing center grouping or edge aggregation.
• The octagon mask derived from extreme points yields 18.9% Mask AP on COCO val2017, substantially better than a naive bounding-box mask and competitive when combined with DEXTR, reaching 34.6% Mask AP.
• Ghost box suppression and edge aggregation contribute modest but meaningful improvements, indicating robustness of the bottom-up grouping approach.