[Paper Review] OneNet: Towards End-to-End One-Stage Object Detection.

Peize Sun, Yi Jiang+4 more|arXiv (Cornell University)|Dec 10, 2020

Advanced Neural Network Applications22 references21 citations

TL;DR

This paper proposes Minimum Cost Assignment, a novel label assignment method that combines classification and location costs to enable end-to-end one-stage object detection without Non-Maximum Suppression (NMS). By assigning only the sample with minimum total cost to each ground-truth, OneNet eliminates redundant high-confidence boxes, achieving 35.0 AP at 80 FPS and 37.7 AP at 50 FPS on COCO with 512×512 input.

ABSTRACT

End-to-end one-stage object detection trailed thus far. This paper discovers that the lack of classification cost between sample and ground-truth in label assignment is the main obstacle for one-stage detectors to remove Non-maximum Suppression(NMS) and reach end-to-end. Existing one-stage object detectors assign labels by only location cost, e.g. box IoU or point distance. Without classification cost, sole location cost leads to redundant boxes of high confidence scores in inference, making NMS necessary post-processing. To design an end-to-end one-stage object detector, we propose Minimum Cost Assignment. The cost is the summation of classification cost and location cost between sample and ground-truth. For each object ground-truth, only one sample of minimum cost is assigned as the positive sample; others are all negative samples. To evaluate the effectiveness of our method, we design an extremely simple one-stage detector named OneNet. Our results show that when trained with Minimum Cost Assignment, OneNet avoids producing duplicated boxes and achieves to end-to-end detector. On COCO dataset, OneNet achieves 35.0 AP/80 FPS and 37.7 AP/50 FPS with image size of 512 pixels. We hope OneNet could serve as an effective baseline for end-to-end one-stage object detection. The code is available at: \url{this https URL}.

Motivation & Objective

• To address the lack of classification cost in label assignment as the key obstacle preventing one-stage detectors from achieving end-to-end performance without NMS.
• To eliminate redundant high-confidence predictions caused by reliance solely on location cost in existing one-stage detectors.
• To design a simple, effective one-stage detector (OneNet) that achieves end-to-end inference by integrating classification and location costs in label assignment.
• To demonstrate that end-to-end one-stage detection is feasible with proper cost-based label assignment, setting a new baseline for future research.

Proposed method

• Proposes Minimum Cost Assignment, where the cost between a sample and ground-truth is the sum of classification cost and location cost.
• Uses classification cost (e.g., cross-entropy) and location cost (e.g., IoU or point distance) to evaluate sample-ground-truth pairs.
• For each ground-truth, assigns only the sample with minimum total cost as positive; all others are labeled negative.
• Introduces OneNet, an extremely simple one-stage detector architecture to evaluate the proposed label assignment method.
• Employs a standard one-stage head with anchor-free design, relying solely on the cost-based assignment for training.
• Eliminates the need for NMS during inference by ensuring only one high-quality prediction per object through cost minimization.

Experimental results

Research questions

• RQ1 Can end-to-end one-stage object detection be achieved without Non-Maximum Suppression (NMS) by improving label assignment?
• RQ2 Does incorporating classification cost into label assignment reduce redundant high-confidence predictions in one-stage detectors?
• RQ3 Can a simple detector like OneNet achieve competitive accuracy while eliminating NMS through cost-based assignment?
• RQ4 How does the combination of classification and location costs in label assignment affect detection performance and inference speed?
• RQ5 Is Minimum Cost Assignment sufficient to enable end-to-end training and inference in one-stage detectors?

Key findings

• OneNet achieves 35.0 AP at 80 FPS and 37.7 AP at 50 FPS on COCO with 512×512 input, demonstrating strong performance without NMS.
• The proposed Minimum Cost Assignment successfully eliminates redundant high-confidence boxes that otherwise necessitate NMS in standard one-stage detectors.
• By integrating classification cost into label assignment, the model avoids overconfidence in multiple predictions per object, enabling end-to-end inference.
• The method enables a simple detector (OneNet) to achieve state-of-the-art performance for end-to-end one-stage detection at inference speed.
• The results validate that the lack of classification cost in label assignment is the primary reason one-stage detectors require NMS, and its inclusion enables end-to-end operation.
• The code is publicly available, providing a strong baseline for future research in end-to-end one-stage object detection.