[Paper Review] Gradient Harmonized Single-stage Detector

Buyu Li, Yu Liu+1 more|arXiv (Cornell University)|Nov 13, 2018

Anomaly Detection Techniques and Applications35 references67 citations

TL;DR

The paper introduces Gradient Harmonizing Mechanism (GHM) to balance gradient contributions in one-stage detectors, proposing GHM-C for classification and GHM-R for regression, achieving state-of-the-art results on COCO without heavy hyper-parameter tuning.

ABSTRACT

Despite the great success of two-stage detectors, single-stage detector is still a more elegant and efficient way, yet suffers from the two well-known disharmonies during training, i.e. the huge difference in quantity between positive and negative examples as well as between easy and hard examples. In this work, we first point out that the essential effect of the two disharmonies can be summarized in term of the gradient. Further, we propose a novel gradient harmonizing mechanism (GHM) to be a hedging for the disharmonies. The philosophy behind GHM can be easily embedded into both classification loss function like cross-entropy (CE) and regression loss function like smooth-$L_1$ ($SL_1$) loss. To this end, two novel loss functions called GHM-C and GHM-R are designed to balancing the gradient flow for anchor classification and bounding box refinement, respectively. Ablation study on MS COCO demonstrates that without laborious hyper-parameter tuning, both GHM-C and GHM-R can bring substantial improvement for single-stage detector. Without any whistles and bells, our model achieves 41.6 mAP on COCO test-dev set which surpasses the state-of-the-art method, Focal Loss (FL) + $SL_1$, by 0.8.

Motivation & Objective

• Identify the source of training disharmony in one-stage detectors (class/attribute imbalance).
• Propose a gradient-based harmonizing mechanism to balance gradient contributions during training.
• Develop GHM-C for classification and GHM-R for regression that adapt to mini-batch data without extensive hyper-parameter tuning.
• Demonstrate improvements on COCO using RetinaNet-style single-stage detectors and compare with focal loss and other baselines.

Proposed method

• Define gradient density GD(g) as the distribution of gradient norms g across training samples.
• Compute a gradient harmonizing parameter beta_i = N / GD(g_i) per sample to reweight loss.
• Formulate GHM-C by replacing standard CE loss with L_GHM-C = (1/N) sum_i beta_i L_CE(p_i, p_i*).
• Extend GHM to regression by introducing ASL1 (Authentic Smooth L1) with gradient gr and applying L_GHM-R = (1/N) sum_i beta_i ASL1(d_i).
• Approximate gradient density using unit regions (epsilon) with EMA smoothing to ensure stability in mini-batch updates.
• Show that GHM adapts to data distribution in each batch and reduces dominance of easy negatives and outliers.

Experimental results

Research questions

• RQ1 Can gradient-density-based reweighting improve the training efficiency and accuracy of one-stage detectors?
• RQ2 How do GHM-C and GHM-R compare to cross-entropy and smooth L1 losses, respectively, on COCO benchmarks?
• RQ3 Does the proposed EMA-based gradient density estimation provide stable and scalable training in large-scale datasets?
• RQ4 Can the GHM approach transfer to two-stage detectors and other backbones while maintaining or improving accuracy?

Key findings

• GHM-C substantially improves classification performance over standard CE and is competitive with, or better than, Focal Loss on COCO.
• GHM-R improves bounding box regression over SL1 and ASL1, particularly at higher IoU thresholds, indicating better localization.
• The combination of GHM-C and GHM-R with RetinaNet achieves state-of-the-art-like results on COCO test-dev, outperforming Focal Loss variants.
• With unit-region approximation (M around 30), training remains efficient and substantially faster than naive densities, while maintaining performance gains.
• GHM approach extends to two-stage detectors, yielding improved AP over SL1 baselines in Faster R-CNN variants.
• On COCO test-dev, GHM-C + GHM-R with RetinaNet-ResNet-101 achieves 39.9 AP and with ResNeXt-101 achieves 41.6 AP, exceeding Focal Loss baselines.