[Paper Review] Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework

Qiang Zhou, Chaohui Yu+3 more|arXiv (Cornell University)|Mar 21, 2021

Advanced Neural Network ApplicationsComputer Science45 references25 citations

TL;DR

Instant-Teaching introduces an end-to-end semi-supervised object detection framework using instant pseudo labeling with extended weak-strong data augmentations and a co-rectify scheme, achieving state-of-the-art results on MS-COCO and PASCAL VOC.

ABSTRACT

Supervised learning based object detection frameworks demand plenty of laborious manual annotations, which may not be practical in real applications. Semi-supervised object detection (SSOD) can effectively leverage unlabeled data to improve the model performance, which is of great significance for the application of object detection models. In this paper, we revisit SSOD and propose Instant-Teaching, a completely end-to-end and effective SSOD framework, which uses instant pseudo labeling with extended weak-strong data augmentations for teaching during each training iteration. To alleviate the confirmation bias problem and improve the quality of pseudo annotations, we further propose a co-rectify scheme based on Instant-Teaching, denoted as Instant-Teaching$^*$. Extensive experiments on both MS-COCO and PASCAL VOC datasets substantiate the superiority of our framework. Specifically, our method surpasses state-of-the-art methods by 4.2 mAP on MS-COCO when using $2\%$ labeled data. Even with full supervised information of MS-COCO, the proposed method still outperforms state-of-the-art methods by about 1.0 mAP. On PASCAL VOC, we can achieve more than 5 mAP improvement by applying VOC07 as labeled data and VOC12 as unlabeled data.

Motivation & Objective

• Motivate reducing dependence on large labeled datasets for object detection by leveraging unlabeled data through semi-supervised learning.
• Develop an end-to-end SSOD framework that updates pseudo labels in real-time during training.
• Improve pseudo-label quality and reduce confirmation bias via augmentation strategies and model cooperation.
• Demonstrate scalability and effectiveness on MS-COCO and PASCAL VOC under various labeling regimes.

Proposed method

• Propose Instant-Teaching, which generates pseudo annotations for unlabeled data within each training iteration using weak augmentation.
• Apply strong augmentations to unlabeled data (including Mixup and Mosaic) to train on pseudo labels.
• Minimize a joint loss comprising supervised detection loss and unsupervised loss from pseudo labels.
• Introduce a co-rectify scheme (Instant-Teaching *) that trains two models with shared structure but different weights to rectify false predictions.
• Use a confidence-based filtering (tau) and NMS to curate pseudo annotations during the pseudo labeling step.
• Extend STAC-style weak-strong augmentation with additional augmentations to improve robustness and pseudo-label quality.

Figure 1: The proposed semi-supervised object detection framework. Instant-Teaching includes instant pseudo labeling with extended weak-strong data augmentations. Instant-Teaching ∗ represents Instant-Teaching combined with our co-rectify scheme.

Experimental results

Research questions

• RQ1 Can an end-to-end SSOD framework update pseudo labels on-the-fly to improve learning from unlabeled data?
• RQ2 Do extended weak-strong augmentations (including Mixup and Mosaic) improve semi-supervised object detection performance?
• RQ3 Does co-rectify help mitigate confirmation bias and further boost detection accuracy in SSOD?
• RQ4 How does Instant-Teaching compare to STAC and other SSOD methods on MS-COCO and PASCAL VOC under various labeling regimes?

Key findings

• Instant-Teaching substantially outperforms STAC across MS-COCO protocols (e.g., 1% labeled data: 18.05 mAP vs 13.97; 2%: 22.45 vs 18.25).
• Instant-Teaching ∗ (with co-rectify) achieves 18.05 to 22.45 mAP across 1%–2% COCO protocols and up to 30.40 mAP on 10% COCO, with 40.20 mAP on full COCO.
• On VOC07, Instant-Teaching ∗ improves from 44.64 to 50.00 mAP when using VOC07 labeled and VOC12 unlabeled data, and reaches 50.80 mAP with MS-COCO unlabeled data.
• Extending strong augmentations with Mixup and Mosaic yields the best 5% COCO result (25.60 mAP) compared to STAC (23.14 mAP).
• Co-rectify accelerates learning and yields higher-quality pseudo annotations by having two models refine each other’s predictions, reducing confirmation bias.

Figure 2: Mixup and Mosaic data augmentations for semi-supervised object detection learning.