[Paper Review] S4OD: Semi-Supervised learning for Single-Stage Object Detection

Yueming Zhang, Xingxu Yao+8 more|arXiv (Cornell University)|Apr 9, 2022

Video Surveillance and Tracking Methods4 citations

TL;DR

This paper proposes S4OD, a semi-supervised learning framework for single-stage object detectors that addresses severe foreground-background class imbalance by introducing a Dynamic Self-Adaptive Threshold (DSAT) for pseudo label selection and a NMS-UNC module to assess regression uncertainty. Using only 10% labeled COCO data, S4OD achieves 35.0% AP on FCOS and 32.9% on RetinaNet, setting new state-of-the-art performance.

ABSTRACT

Single-stage detectors suffer from extreme foreground-background class imbalance, while two-stage detectors do not. Therefore, in semi-supervised object detection, two-stage detectors can deliver remarkable performance by only selecting high-quality pseudo labels based on classification scores. However, directly applying this strategy to single-stage detectors would aggravate the class imbalance with fewer positive samples. Thus, single-stage detectors have to consider both quality and quantity of pseudo labels simultaneously. In this paper, we design a dynamic self-adaptive threshold (DSAT) strategy in classification branch, which can automatically select pseudo labels to achieve an optimal trade-off between quality and quantity. Besides, to assess the regression quality of pseudo labels in single-stage detectors, we propose a module to compute the regression uncertainty of boxes based on Non-Maximum Suppression. By leveraging only 10% labeled data from COCO, our method achieves 35.0% AP on anchor-free detector (FCOS) and 32.9% on anchor-based detector (RetinaNet).

Motivation & Objective

• To address the severe foreground-background class imbalance in single-stage detectors during semi-supervised learning.
• To develop a dynamic, adaptive strategy for selecting high-quality and sufficient-volume pseudo labels without manual hyperparameter tuning.
• To enable reliable assessment of regression quality in single-stage detectors for pseudo label filtering.
• To achieve state-of-the-art performance in semi-supervised single-stage object detection with minimal labeled data.

Proposed method

• Proposes a Dynamic Self-Adaptive Threshold (DSAT) that automatically adjusts the classification score threshold for pseudo label selection based on model performance, balancing quality and quantity.
• Introduces the NMS-UNC module, which computes regression uncertainty by measuring the standard deviation of redundant boxes in Non-Maximum Suppression (NMS).
• Uses the uncertainty score from NMS-UNC to filter out low-quality regression pseudo labels, improving training stability.
• Integrates DSAT and NMS-UNC into a semi-supervised training pipeline that leverages both labeled and unlabeled data.
• Employs a two-stage training process: first, a teacher model generates pseudo labels; second, a student model is trained with both labeled data and filtered pseudo labels.
• Uses F1-score-based dynamic thresholding in DSAT to adaptively select pseudo labels based on the harmonic mean of precision and recall during training.

Experimental results

Research questions

• RQ1 How can we balance the trade-off between quality and quantity of pseudo labels in semi-supervised single-stage object detection?
• RQ2 Can a dynamic, self-adaptive threshold strategy outperform fixed threshold ablation studies in pseudo label selection?
• RQ3 How can regression uncertainty be effectively measured and utilized for filtering pseudo labels in single-stage detectors?
• RQ4 Does the proposed method achieve state-of-the-art performance with only 10% labeled data on COCO?

Key findings

• S4OD achieves 35.0% AP on the anchor-free FCOS detector and 32.9% AP on the anchor-based RetinaNet using only 10% labeled COCO data.
• The DSAT strategy outperforms the best fixed threshold (0.9) by +1.8% AP, demonstrating its superiority in adaptive threshold selection.
• The optimal uncertainty threshold for NMS-UNC is found to be 0.08, yielding the highest AP of 31.3 with 49.1 AP@0.5 and 33.0 AP@0.75.
• The optimal score threshold for NMS-UNC is 0.4, which balances computational efficiency and performance, outperforming lower thresholds like 0.3 and higher ones like 0.5.
• The addition of both DSAT and NMS-UNC increases training time by only 5.55%, making the method practical for real-world deployment.
• The DSAT and NMS-UNC modules are model-agnostic and can be applied to various single-stage detectors without architectural modification.