[Paper Review] Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection

Shifeng Zhang, Cheng Chi+3 more|arXiv (Cornell University)|Dec 5, 2019

Advanced Neural Network Applications70 references107 citations

TL;DR

The paper shows that the key gap between anchor-based and anchor-free detectors lies in how positives/negatives are defined; it introduces Adaptive Training Sample Selection (ATSS) to automatically determine positives/negatives, improving both paradigms and reducing the need for multiple anchors per location.

ABSTRACT

Object detection has been dominated by anchor-based detectors for several years. Recently, anchor-free detectors have become popular due to the proposal of FPN and Focal Loss. In this paper, we first point out that the essential difference between anchor-based and anchor-free detection is actually how to define positive and negative training samples, which leads to the performance gap between them. If they adopt the same definition of positive and negative samples during training, there is no obvious difference in the final performance, no matter regressing from a box or a point. This shows that how to select positive and negative training samples is important for current object detectors. Then, we propose an Adaptive Training Sample Selection (ATSS) to automatically select positive and negative samples according to statistical characteristics of object. It significantly improves the performance of anchor-based and anchor-free detectors and bridges the gap between them. Finally, we discuss the necessity of tiling multiple anchors per location on the image to detect objects. Extensive experiments conducted on MS COCO support our aforementioned analysis and conclusions. With the newly introduced ATSS, we improve state-of-the-art detectors by a large margin to $50.7\%$ AP without introducing any overhead. The code is available at https://github.com/sfzhang15/ATSS

Motivation & Objective

• Identify the essential differences between anchor-based and anchor-free detectors in training sample definition.
• Propose an adaptive, largely hyperparameter-free method to select positive/negative samples.
• Demonstrate that tiling multiple anchors per location is unnecessary.
• Show that ATSS improves both anchor-based and anchor-free detectors on MS COCO without extra computational cost.

Proposed method

• Define candidate positives per ground-truth by selecting k closest anchors to the ground-truth center on each pyramid level.
• Compute IoU between candidates and the ground-truth, derive mean m_g and std v_g, set adaptive threshold t_g = m_g + v_g.
• Select positives as candidates with IoU >= t_g and centers inside the ground-truth box; negatives are remaining anchors.
• Apply ATSS to replace fixed positive-sample rules in RetinaNet (#A=1) and to modify selection in FCOS (lite and full versions).
• Report that ATSS requires only one hyperparameter k and is robust to its settings.

Experimental results

Research questions

• RQ1 What is the essential factor differentiating anchor-based and anchor-free detectors in training?
• RQ2 Can an adaptive, sample-centric strategy improve detection performance across both detector families?
• RQ3 Is tiling multiple anchors per location necessary for strong performance?

Key findings

• ATSS bridges the performance gap by adaptively selecting positives/negatives based on object statistics, achieving higher AP on COCO without overhead.
• Replacing fixed sample rules with ATSS yields improvements for RetinaNet (#A=1) across AP metrics (e.g., AP up to 39.3 from 37.0).
• For FCOS, ATSS (full version) consistently improves AP over the baseline and the lite version, with larger gains when using the full approach.
• ATSS with various backbones (ResNet-101 to ResNeXt-64x4d-101 with DCN) achieves state-of-the-art results, e.g., up to 50.7% AP on COCO test-dev under multi-scale testing.
• ATSS demonstrates robustness to anchor size and aspect ratio variations, and the method reduces the need for multiple anchors per location.
• Hyperparameter k is shown to be robust in a wide range (roughly 7–17) with smaller or larger values providing diminishing or negative returns.