[Paper Review] Unbiased Teacher v2: Semi-supervised Object Detection for Anchor-free and Anchor-based Detectors

Yen‐Cheng Liu, Chih‐Yao Ma+1 more|arXiv (Cornell University)|Jun 19, 2022

Advanced Image and Video Retrieval Techniques4 citations

TL;DR

This paper proposes Unbiased Teacher v2, a semi-supervised object detection framework that generalizes to both anchor-free and anchor-based detectors by introducing Listen2Student—a novel uncertainty-based pseudo-label selection mechanism for regression heads. It achieves state-of-the-art performance on COCO-standard, COCO-additional, and VOC benchmarks by reducing misleading pseudo-labels through relative uncertainty comparison between Teacher and Student networks.

ABSTRACT

With the recent development of Semi-Supervised Object Detection (SS-OD) techniques, object detectors can be improved by using a limited amount of labeled data and abundant unlabeled data. However, there are still two challenges that are not addressed: (1) there is no prior SS-OD work on anchor-free detectors, and (2) prior works are ineffective when pseudo-labeling bounding box regression. In this paper, we present Unbiased Teacher v2, which shows the generalization of SS-OD method to anchor-free detectors and also introduces Listen2Student mechanism for the unsupervised regression loss. Specifically, we first present a study examining the effectiveness of existing SS-OD methods on anchor-free detectors and find that they achieve much lower performance improvements under the semi-supervised setting. We also observe that box selection with centerness and the localization-based labeling used in anchor-free detectors cannot work well under the semi-supervised setting. On the other hand, our Listen2Student mechanism explicitly prevents misleading pseudo-labels in the training of bounding box regression; we specifically develop a novel pseudo-labeling selection mechanism based on the Teacher and Student's relative uncertainties. This idea contributes to favorable improvement in the regression branch in the semi-supervised setting. Our method, which works for both anchor-free and anchor-based methods, consistently performs favorably against the state-of-the-art methods in VOC, COCO-standard, and COCO-additional.

Motivation & Objective

• To address the lack of semi-supervised object detection (SS-OD) methods for anchor-free detectors, which are increasingly popular in object detection.
• To resolve the issue of ineffective pseudo-labeling in bounding box regression under semi-supervised settings, especially when using confidence thresholding.
• To improve regression performance by reducing misleading pseudo-labels through a more robust selection mechanism than confidence-based thresholding.
• To bridge the performance gap between anchor-free and anchor-based detectors under semi-supervised learning by applying a unified, effective method.
• To empirically validate the effectiveness of the proposed method across diverse benchmarks including COCO-standard, COCO-additional, and VOC.

Proposed method

• Introduces Listen2Student, a mechanism that selects regression pseudo-labels based on relative uncertainty between Teacher and Student networks.
• Uses boundary-wise uncertainty estimation for each of the four bounding box coordinates to guide pseudo-label selection, rather than relying on classification confidence.
• Enforces unsupervised regression loss only on pseudo-labels where the Teacher has lower uncertainty than the Student, reducing noise from unreliable predictions.
• Applies this mechanism to both anchor-free (e.g., FCOS, RetinaNet) and anchor-based (e.g., Faster R-CNN) detectors, ensuring generalization across architectures.
• Employs a consistent self-training framework with data augmentation and consistency regularization, with pseudo-labeling applied only to high-confidence, low-uncertainty regions.
• Uses a multi-stage training protocol with iterative pseudo-labeling and model refinement, maintaining consistency between Teacher and Student predictions.

Figure 1 : To improve the unsupervised regression loss, we propose (a) Listen2Student , which explicitly compares the prediction uncertainties between the Teacher and the Student and selects these instances where the teacher has lower uncertainty than the student. We then enforce the unsupervised re

Experimental results

Research questions

• RQ1 Why do existing SS-OD methods perform poorly on anchor-free detectors compared to anchor-based ones?
• RQ2 How does the reliability of centerness and localization-based labeling degrade under semi-supervised settings?
• RQ3 Can relative uncertainty between Teacher and Student improve pseudo-label quality for bounding box regression?
• RQ4 Does a boundary-wise uncertainty estimation mechanism lead to better regression performance than confidence-based thresholding?
• RQ5 Can a unified SS-OD framework achieve state-of-the-art results across both anchor-free and anchor-based detectors?

Key findings

• Unbiased Teacher v2 achieves 44.75 mAP on COCO-additional using only COCO2017-train as labeled data and COCO2017-unlabeled as unlabeled data, outperforming the supervised baseline of 40.90 mAP.
• On VOC with VOC07 as labeled and VOC12+COCO20cls as unlabeled, the model reaches 58.08 mAP, surpassing the supervised baseline and demonstrating strong generalization.
• The Listen2Student mechanism improves mAP across all IoU thresholds (AP55 to AP95), with the most significant gains on stricter metrics like AP95, confirming better boundary precision.
• Confidence thresholding alone degrades performance on AP95, indicating that it introduces misleading pseudo-labels that harm precise localization.
• The method reduces performance gaps between anchor-free and anchor-based detectors under semi-supervised learning, showing that SS-OD can be effectively generalized to modern anchor-free architectures.
• Ablation studies confirm that relative uncertainty-based selection is more effective than confidence-based selection, especially in high-precision detection scenarios.

Figure 2 : Illustration of Centerness bias issue. (a) Selecting pseudo-boxes based on box scores leads to worse results in semi-supervised learning compared with selecting based on classification scores. (b) Box scores of the anchor-free detectors [ 29 , 37 ] are defined as the multiplication of the