[Paper Review] Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection

Li Xiang, Wenhai Wang+6 more|arXiv (Cornell University)|Jun 8, 2020

Advanced Neural Network Applications764 citations

TL;DR

This paper introduces Generalized Focal Loss (GFL) which unifies localization quality with classification and models bounding boxes as arbitrary distributions, achieving state-of-the-art COCO results with efficient inference.

ABSTRACT

One-stage detector basically formulates object detection as dense classification and localization. The classification is usually optimized by Focal Loss and the box location is commonly learned under Dirac delta distribution. A recent trend for one-stage detectors is to introduce an individual prediction branch to estimate the quality of localization, where the predicted quality facilitates the classification to improve detection performance. This paper delves into the representations of the above three fundamental elements: quality estimation, classification and localization. Two problems are discovered in existing practices, including (1) the inconsistent usage of the quality estimation and classification between training and inference and (2) the inflexible Dirac delta distribution for localization when there is ambiguity and uncertainty in complex scenes. To address the problems, we design new representations for these elements. Specifically, we merge the quality estimation into the class prediction vector to form a joint representation of localization quality and classification, and use a vector to represent arbitrary distribution of box locations. The improved representations eliminate the inconsistency risk and accurately depict the flexible distribution in real data, but contain continuous labels, which is beyond the scope of Focal Loss. We then propose Generalized Focal Loss (GFL) that generalizes Focal Loss from its discrete form to the continuous version for successful optimization. On COCO test-dev, GFL achieves 45.0\\% AP using ResNet-101 backbone, surpassing state-of-the-art SAPD (43.5\\%) and ATSS (43.6\\%) with higher or comparable inference speed, under the same backbone and training settings. Notably, our best model can achieve a single-model single-scale AP of 48.2\\%, at 10 FPS on a single 2080Ti GPU. Code and models are available at https://github.com/implus/GFocal.

Motivation & Objective

• Address inconsistency between localization quality estimation and classification during training and testing in one-stage detectors.
• Propose joint representations for localization quality and class scores to unify training and inference.
• Model bounding boxes as flexible distributions rather than Dirac delta or simple Gaussians to capture uncertainty.
• Develop Generalized Focal Loss (GFL) that extends Focal Loss to continuous targets for both quality estimation and distribution learning.
• Demonstrate state-of-the-art performance on COCO with efficient inference using GFL with ATSS/ResNet backbones.

Proposed method

• Introduce a joint classification-IoU representation where the ground-truth category index holds the localization quality (IoU) value, creating a unified score used end-to-end.
• Extend Focal Loss to continuous targets by proposing Quality Focal Loss (QFL) with a continuous target y in [0,1] and a modulating factor |y - p|^β.
• Represent bounding box locations as a General distribution P(x) over a discretized space and predict it with a softmax, enabling a learnable, flexible regression target.
• Introduce Distribution Focal Loss (DFL) to encourage probability mass near the target binding coordinates, focusing learning around y_i and y_{i+1} closest to the target.
• Combine QFL and DFL within Generalized Focal Loss (GFL), with a joint loss L = L_QFL + L_DFL + L_B terms, trained on dense locations across feature pyramids.
• Demonstrate compatibility with standard one-stage detectors and show negligible inference overhead.

Experimental results

Research questions

• RQ1 Can unifying localization quality with classification reduce training-test inconsistency in one-stage detectors?
• RQ2 Does modeling bounding boxes as flexible, learnable distributions improve localization accuracy under real-world ambiguity?
• RQ3 How can Focal Loss be generalized to continuous targets to optimize both quality estimation and distribution learning?
• RQ4 What is the impact of jointly applying QFL and DFL on COCO AP compared with baseline and other state-of-the-art detectors?
• RQ5 Is the proposed GFL approach computationally efficient enough for practical deployment?

Key findings

• GFL with a joint classification-IoU representation yields higher AP than baselines across ablations on COCO (e.g., surpassing SAPD and ATSS under the same backbone and training settings).
• QFL effectively learns continuous quality estimates per class, improving localization-quality correlation and overall detection performance.
• DFL learns flexible bounding-box distributions, focusing probabilities around the target coordinates and improving box accuracy, particularly under uncertainty.
• Joint use of QFL and DFL provides orthogonal gains and yields about 1 percentage point AP improvement over strong ATSS baselines, with negligible inference overhead.
• GFL achieves 45.0% AP on COCO test-dev with ResNet-101 backbone, and 48.2% single-model single-scale AP at 10 FPS on a single 2080Ti GPU, outperforming several prior methods.