[Paper Review] Distilling Object Detectors with Feature Richness

Zhixing Du, Rui Zhang+5 more|arXiv (Cornell University)|Nov 1, 2021

Advanced Neural Network ApplicationsComputer Science31 references36 citations

TL;DR

The paper introduces Feature Richness Score (FRS) to select informative features for distilling object detectors, leveraging outside bounding boxes and pruning inside-box misclassifications, and shows improvements across anchor-based, anchor-free, and two-stage detectors.

ABSTRACT

In recent years, large-scale deep models have achieved great success, but the huge computational complexity and massive storage requirements make it a great challenge to deploy them in resource-limited devices. As a model compression and acceleration method, knowledge distillation effectively improves the performance of small models by transferring the dark knowledge from the teacher detector. However, most of the existing distillation-based detection methods mainly imitating features near bounding boxes, which suffer from two limitations. First, they ignore the beneficial features outside the bounding boxes. Second, these methods imitate some features which are mistakenly regarded as the background by the teacher detector. To address the above issues, we propose a novel Feature-Richness Score (FRS) method to choose important features that improve generalized detectability during distilling. The proposed method effectively retrieves the important features outside the bounding boxes and removes the detrimental features within the bounding boxes. Extensive experiments show that our methods achieve excellent performance on both anchor-based and anchor-free detectors. For example, RetinaNet with ResNet-50 achieves 39.7% in mAP on the COCO2017 dataset, which even surpasses the ResNet-101 based teacher detector 38.9% by 0.8%. Our implementation is available at https://github.com/duzhixing/FRS.

Motivation & Objective

• Motivate knowledge distillation for object detectors with a focus on informative features beyond bounding boxes.
• Propose Feature Richness Score (FRS) to identify object-like features across all regions, not just inside boxes.
• Demonstrate a plug-and-play distillation framework applicable to both anchor-based/anchor-free and two-stage detectors.
• Show that FRS improves generalized detectability by leveraging outside-box information and reducing within-box misclassified features.

Proposed method

• Define feature richness S as the max over categories of P(c|f, theta) using teacher classification scores.
• Compute per-pyramid-level feature richness masks S_l from corresponding teacher classification scores.
• Distill both FPN layers (L_FPN) and the classification head (L_head) using masks to weight pixel-wise distillation losses.
• Combine losses with standard GT loss: L = L_GT + alpha L_FPN + beta L_head.

Experimental results

Research questions

• RQ1 Can feature-rich regions outside bounding boxes provide beneficial guidance for distillation of object detectors?
• RQ2 Does weighting distillation by a pixel-wise feature richness mask improve student performance across detector types?
• RQ3 How does FRS perform across anchor-based, anchor-free, and two-stage detectors on COCO?
• RQ4 To what extent do outside-box features and inside-box mislabeled features affect distillation quality?

Key findings

• FRS yields improvements across RetinaNet, GFL, FCOS, and Faster R-CNN withResNet-50 as student and ResNet-101 as teacher.
• RetinaNet-Res50 with 2x schedule achieves 39.7% mAP, surpassing the teacher (38.9%) by 0.8%.
• GFL-Res50 (1x) improves by 3.4% mAP over baseline; with 2x in some setups gains exceed 1.8–4.2% across metrics.
• FCOS-Res50 (2x) achieves 40.9% mAP, surpassing the teacher's performance.
• Ablation shows both FPN and classification head distillation contribute gains, with combined improvements up to 2.3% mAP.
• Qualitative and entropy analyses indicate TP+FP regions (outside-box informative areas) are particularly beneficial for distillation.