[Paper Review] Towards End-to-end Semi-supervised Learning for One-stage Object Detection

Gen Luo, Yiyi Zhou+3 more|arXiv (Cornell University)|Feb 22, 2023

Advanced Neural Network Applications4 citations

TL;DR

This paper proposes OneTeacher, a novel teacher-student framework for end-to-end semi-supervised object detection in one-stage detectors like YOLOv5. It addresses low-quality pseudo-labeling and multi-task optimization conflicts via Multi-view Pseudo-label Refinement (MPR) and Decoupled Semi-supervised Optimization (DSO), achieving 15.0% relative AP gain over Unbiased Teacher on COCO.

ABSTRACT

Semi-supervised object detection (SSOD) is a research hot spot in computer vision, which can greatly reduce the requirement for expensive bounding-box annotations. Despite great success, existing progress mainly focuses on two-stage detection networks like FasterRCNN, while the research on one-stage detectors is often ignored. In this paper, we focus on the semi-supervised learning for the advanced and popular one-stage detection network YOLOv5. Compared with Faster-RCNN, the implementation of YOLOv5 is much more complex, and the various training techniques used in YOLOv5 can also reduce the benefit of SSOD. In addition to this challenge, we also reveal two key issues in one-stage SSOD, which are low-quality pseudo-labeling and multi-task optimization conflict, respectively. To address these issues, we propose a novel teacher-student learning recipe called OneTeacher with two innovative designs, namely Multi-view Pseudo-label Refinement (MPR) and Decoupled Semi-supervised Optimization (DSO). In particular, MPR improves the quality of pseudo-labels via augmented-view refinement and global-view filtering, and DSO handles the joint optimization conflicts via structure tweaks and task-specific pseudo-labeling. In addition, we also carefully revise the implementation of YOLOv5 to maximize the benefits of SSOD, which is also shared with the existing SSOD methods for fair comparison. To validate OneTeacher, we conduct extensive experiments on COCO and Pascal VOC. The extensive experiments show that OneTeacher can not only achieve superior performance than the compared methods, e.g., 15.0% relative AP gains over Unbiased Teacher, but also well handle the key issues in one-stage SSOD. Our source code is available at: https://github.com/luogen1996/OneTeacher.

Motivation & Objective

• Address the underexplored challenge of semi-supervised object detection in one-stage detectors like YOLOv5, which are widely used but often overlooked in existing SSOD research.
• Identify and resolve two key issues in one-stage SSOD: low-quality pseudo-labeling due to dense prediction and multi-task optimization conflicts from shared label criteria.
• Maximize the benefits of teacher-student learning in YOLOv5 by carefully revising its implementation to align with SSOD best practices.
• Propose a new training recipe that enables end-to-end semi-supervised learning for one-stage detectors, improving both performance and robustness.

Proposed method

• Introduce Multi-view Pseudo-label Refinement (MPR), which enhances pseudo-label quality through augmented-view comparisons and global-view filtering to reduce noisy predictions.
• Implement Decoupled Semi-supervised Optimization (DSO) by modifying the network structure to decouple classification and regression heads, enabling task-specific pseudo-labeling with different confidence thresholds.
• Apply instance-level refinement via cross-view consistency checks between original and augmented views to improve robustness of pseudo-labels.
• Use image-level predictions to filter out low-confidence pseudo-labels, reducing false positives and improving overall label quality.
• Adapt YOLOv5’s training pipeline by incorporating advanced data augmentations and optimization techniques (e.g., cosine decay, EMA) to maximize SSOD performance.
• Deploy separate pseudo-labeling thresholds for classification (0.4) and regression (0.4), reflecting different noise tolerances in one-stage detectors.

Figure 1 : Qualities of pseudo-labels of YOLOv5 [ 22 ] and FasterRCNN [ 7 ] in Unbiased Teacher [ 11 ] . The subplots of (a) and (b) show that the average pseudo-label quality of YOLOv5 is much lower than that of FasterRCNN, especially in the initial training phase. Subplot (c) illustrates the diffe

Experimental results

Research questions

• RQ1 Why do existing teacher-student methods for semi-supervised object detection underperform on one-stage detectors like YOLOv5 compared to two-stage models?
• RQ2 How can pseudo-label quality be improved in one-stage detectors, which suffer from high noise due to dense prediction across thousands of anchors?
• RQ3 To what extent do multi-task optimization conflicts between classification and regression degrade performance in one-stage SSOD, and how can they be mitigated?
• RQ4 What is the optimal configuration of data augmentations and pseudo-labeling thresholds for YOLOv5 in a semi-supervised setting?
• RQ5 Can a unified training recipe be designed to maximize the benefits of semi-supervised learning in one-stage detectors without compromising their inherent architecture?

Key findings

• OneTeacher achieves a 15.0% relative AP gain over Unbiased Teacher on COCO under standard 1× setting, demonstrating superior performance in one-stage SSOD.
• The proposed MPR mechanism significantly improves pseudo-label quality, as visualized by reduced false positives and better detection of previously missed objects.
• DSO effectively resolves multi-task optimization conflicts by decoupling classification and regression heads, leading to faster convergence and better performance.
• Using YOLOv5-specific data augmentations instead of standard two-stage SSOD schemes improves mAP by 13.5 points, highlighting the importance of model-specific implementation.
• The optimal pseudo-labeling threshold for YOLOv5 is 0.4 for both classification and regression, which is significantly lower than the 0.7 used in FasterRCNN, indicating higher sensitivity to noise in one-stage detectors.
• Ablation studies confirm that augmented-view refinement in MPR outperforms Flip Ensemble by avoiding noise amplification through unfiltered averaging.

Figure 2 : The framework of OneTeacher. The teacher network produces high-quality pseudo-labels for the student by the proposed Multi-view Pseudo-label Refinement (MPR). Its parameters are updated from the student ones via EMA [ 14 ] . OneTeacher also deploys a novel Decoupled-Semi-supervised Optimi