[Paper Review] Distilling Object Detectors via Decoupled Features

Jianyuan Guo, Kai Han+5 more|arXiv (Cornell University)|Mar 26, 2021

Advanced Neural Network ApplicationsComputer Science65 references26 citations

TL;DR

This paper introduces DeFeat, a knowledge distillation method for object detectors that decouples feature regions (object vs background) and positive vs negative RoI proposals to better transfer knowledge from a teacher to a student, boosting performance on COCO and VOC across one- and two-stage detectors.

ABSTRACT

Knowledge distillation is a widely used paradigm for inheriting information from a complicated teacher network to a compact student network and maintaining the strong performance. Different from image classification, object detectors are much more sophisticated with multiple loss functions in which features that semantic information rely on are tangled. In this paper, we point out that the information of features derived from regions excluding objects are also essential for distilling the student detector, which is usually ignored in existing approaches. In addition, we elucidate that features from different regions should be assigned with different importance during distillation. To this end, we present a novel distillation algorithm via decoupled features (DeFeat) for learning a better student detector. Specifically, two levels of decoupled features will be processed for embedding useful information into the student, i.e., decoupled features from neck and decoupled proposals from classification head. Extensive experiments on various detectors with different backbones show that the proposed DeFeat is able to surpass the state-of-the-art distillation methods for object detection. For example, DeFeat improves ResNet50 based Faster R-CNN from 37.4% to 40.9% mAP, and improves ResNet50 based RetinaNet from 36.5% to 39.7% mAP on COCO benchmark. Our implementation is available at https://github.com/ggjy/DeFeat.pytorch.

Motivation & Objective

• Motivate improved knowledge transfer for object detectors by considering both object and background regions in intermediate features.
• Show that background regions can complement object regions for distillation, challenging the assumption that background is useless.
• Propose a decoupled-feature distillation framework (DeFeat) operating on neck (FPN) features and RoI-aligned proposals.
• Demonstrate effectiveness across two-stage and one-stage detectors and multiple backbones on COCO and VOC datasets.

Proposed method

• Decouple intermediate FPN features into object and background regions using ground-truth masks and apply separate imitation losses (Eq. 5).
• Decouple region proposals in the classification head into positive (object) and negative (background) proposals and distill them with separate KL-divergence losses (Eq. 8, Eq. 9).
• Combine decoupled feature distillation (L_fea), decoupled classification distillation (L_cls), and standard detection losses (L_reg, L_rpn) for end-to-end training (Eq. 3).
• Adopt a teacher–student setup with adaptive weighting (α_obj, α_bg, β_obj, β_bg) and temperature scaling (T_obj, T_bg) to balance gradient magnitudes (Eq. 5, Eq. 8).
• Demonstrate applicability to both Faster R-CNN/FPN (two-stage) and RetinaNet (one-stage) on COCO and VOC.

Experimental results

Research questions

• RQ1 Can background regions in neck features contribute to more effective knowledge distillation for object detectors?
• RQ2 Does decoupling object/positive and background/negative proposals in the classification head improve KD performance for detection?
• RQ3 Is DeFeat applicable across both two-stage and one-stage detectors with different backbones?
• RQ4 What is the quantitative impact of decoupled features on COCO and VOC benchmarks compared to prior KD methods?

Key findings

• DeFeat improves Faster R-CNN with ResNet50-FPN from 37.4% to 40.9% mAP on COCO, and RetinaNet with ResNet50-FPN from 36.5% to 39.7% mAP on COCO.
• Decoupled neck features (object and background) yield a gain over decoupled proposals alone, with backbone distillation contributing to the best results (40.9% mAP on COCO).
• Decoupled region proposals (positive vs negative) in the classification head provide better gradient balance and higher mAP than treating all proposals equally (e.g., 40.9% vs 40.5% under certain configurations).
• On Pascal VOC, DeFeat with teacher–student setups yields 82.28% mAP (vs 80.53% for the baseline student).
• DeFeat consistently outperforms FGFI and TADF baselines on COCO across multiple settings (e.g., 39.7–40.9% mAP for RetinaNet/FPN variants).
• Ablation studies show object regions drive localization and background regions reduce false positives, both contributing to overall gains.