[Paper Review] FCOS: Fully Convolutional One-Stage Object Detection

Zhi Tian, Chunhua Shen+2 more|arXiv (Cornell University)|Apr 2, 2019

Advanced Neural Network Applications32 references799 citations

TL;DR

FCOS presents an anchor- and proposal-free, fully convolutional one-stage object detector that uses multi-level prediction and a center-ness branch to achieve state-of-the-art results among single-model one-stage detectors, e.g., 44.7 AP on COCO with improvements.

ABSTRACT

We propose a fully convolutional one-stage object detector (FCOS) to solve object detection in a per-pixel prediction fashion, analogue to semantic segmentation. Almost all state-of-the-art object detectors such as RetinaNet, SSD, YOLOv3, and Faster R-CNN rely on pre-defined anchor boxes. In contrast, our proposed detector FCOS is anchor box free, as well as proposal free. By eliminating the predefined set of anchor boxes, FCOS completely avoids the complicated computation related to anchor boxes such as calculating overlapping during training. More importantly, we also avoid all hyper-parameters related to anchor boxes, which are often very sensitive to the final detection performance. With the only post-processing non-maximum suppression (NMS), FCOS with ResNeXt-64x4d-101 achieves 44.7% in AP with single-model and single-scale testing, surpassing previous one-stage detectors with the advantage of being much simpler. For the first time, we demonstrate a much simpler and flexible detection framework achieving improved detection accuracy. We hope that the proposed FCOS framework can serve as a simple and strong alternative for many other instance-level tasks. Code is available at:Code is available at: https://tinyurl.com/FCOSv1

Motivation & Objective

• Reformulate object detection as per-pixel prediction similar to semantic segmentation.
• Eliminate anchor boxes and associated hyper-parameters to simplify training and inference.
• Leverage multi-level FPN to improve recall and reduce ambiguity from overlapping boxes.
• Introduce a center-ness branch to suppress low-quality detections and boost accuracy.
• Demonstrate strong performance as a baseline and extension for other instance-wise tasks.

Proposed method

• Predict a 4D vector (l, t, r, b) at each pixel to encode bounding box sides relative to the location.
• Use multi-level feature maps (P3–P7) from FPN to cover object scales and restrict regression per level with m_i thresholds.
• Apply focal loss for classification and IoU-based loss for regression, with normalization and a trainable per-level exponential base.
• Introduce a single-layer center-ness branch that scores how close a pixel is to the object center and multiply this score with classification confidence at inference.
• Train with positive samples as pixels inside ground-truth boxes and use a simple ambiguity rule (smallest-area box for overlaps).
• Employ GN in added heads for stability and reuse RetinaNet-like training/testing hyper-parameters for fair comparison.

Experimental results

Research questions

• RQ1 Can object detection be effectively solved in a per-pixel, anchor-free framework akin to semantic segmentation?
• RQ2 Does removing anchor boxes and related hyper-parameters improve simplicity without sacrificing accuracy?
• RQ3 Can multi-level prediction via FPN alleviate issues of recall and ambiguity due to overlapping ground-truth boxes in an anchor-free detector?
• RQ4 Does a center-ness signal effectively suppress low-quality detections and improve overall performance?
• RQ5 Can FCOS serve as a strong baseline and as a region proposal network for predicate two-stage detectors?

Key findings

• FCOS achieves competitive and often superior performance to anchor-based detectors under the same training/testing settings (e.g., AP comparable or higher than RetinaNet).
• For a ResNet-50 backbone with FPN, FCOS reaches higher AR than some anchor-based counterparts under identical settings.
• Without center-ness, FCOS shows AP around 33.5–36.3 depending on configuration; with center-ness, AP improves to 37.1 (and up to 38.1–38.6 with enhancements).
• Best reported single-model single-scale results reach 44.7 AP with improvements on ResNeXt-64x4d-101-FPN, surpassing the anchor-based RetinaNet by 2.4 AP on COCO test-dev.
• Multi-level prediction reduces ambiguous sample rate from 23.16% (no FPN) to 7.14% (with FPN), and near 1.5% when considering only cross-category overlaps.
• FCOS demonstrates strong potential as a simple, flexible baseline for instance-level tasks and as an effective RPN alternative for two-stage detectors (e.g., AR gains of 8.1% at AR100).