[Paper Review] FCOS: A simple and strong anchor-free object detector

Zhi Tian, Chunhua Shen+2 more|arXiv (Cornell University)|Jun 14, 2020

Advanced Neural Network Applications57 references38 citations

TL;DR

FCOS presents a fully convolutional, anchor-free one-stage object detector using per-pixel predictions, a center-ness branch, and multi-level FPN to achieve competitive accuracy without anchor boxes.

ABSTRACT

In computer vision, object detection is one of most important tasks, which underpins a few instance-level recognition tasks and many downstream applications. Recently one-stage methods have gained much attention over two-stage approaches due to their simpler design and competitive performance. Here we propose a fully convolutional one-stage object detector (FCOS) to solve object detection in a per-pixel prediction fashion, analogue to other dense prediction problems such as semantic segmentation. Almost all state-of-the-art object detectors such as RetinaNet, SSD, YOLOv3, and Faster R-CNN rely on pre-defined anchor boxes. In contrast, our proposed detector FCOS is anchor box free, as well as proposal free. By eliminating the pre-defined set of anchor boxes, FCOS completely avoids the complicated computation related to anchor boxes such as calculating the intersection over union (IoU) scores during training. More importantly, we also avoid all hyper-parameters related to anchor boxes, which are often sensitive to the final detection performance. With the only post-processing non-maximum suppression (NMS), we demonstrate a much simpler and flexible detection framework achieving improved detection accuracy. We hope that the proposed FCOS framework can serve as a simple and strong alternative for many other instance-level tasks. Code and pre-trained models are available at: https://git.io/AdelaiDet

Motivation & Objective

• Motivate removing anchor boxes from modern detectors to simplify design and training.
• Propose a per-pixel prediction framework for object detection analogous to FCNs in segmentation.
• Introduce a center-ness mechanism to suppress low-quality detections.
• Leverage multi-level feature pyramids to improve recall and reduce ambiguity from overlaps.

Proposed method

• Predict a 4D bounding-box vector (l, t, r, b) at each foreground pixel.
• Use a 80-class binary classification head with focal loss for detection without anchors.
• Train with a regression target scaled by the feature level stride and apply GIoU loss for box regression.
• Introduce a center-ness branch that weighs the final scores by distance to object center.
• Employ multi-level prediction (P3–P7) from FPN to handle objects of different scales and reduce overlap ambiguity.
• During inference, score per location is s = sqrt(p * centerness).

Experimental results

Research questions

• RQ1 Can object detection be effectively performed in a fully anchor-free, per-pixel prediction framework?
• RQ2 Does multi-level prediction (FPN) alleviate ambiguity from overlapping ground-truth boxes in an anchor-free detector?
• RQ3 Does a center-ness branch effectively suppress low-quality detections and improve AP?
• RQ4 How does FCOS compare to anchor-based detectors like RetinaNet in accuracy and efficiency on COCO?

Key findings

• Achieves competitive AP on COCO compared to anchor-based detectors (e.g., FCOS with center-ness reaches 38.9 AP).
• Without center-ness, FCOS already beats RetinaNet with 9 anchors per location in AP (38.0 vs. 35.9 with A=9); center-ness further improves to 38.9 AP.
• Multi-level prediction with FPN reduces ambiguous samples to a small fraction (as low as 2.66% with center sampling), mitigating overlap issues.
• Best results on COCO val2017 show FCOS surpasses RetinaNet variants across multiple metrics (AP, AP50, AP75, AR at various thresholds).
• Center-ness weighting substantially improves precision by down-weighting high-class-confidence, low-overlap boxes.