[Paper Review] A Survey of Self-Supervised and Few-Shot Object Detection

Gabriel Huang, Issam Laradji+3 more|arXiv (Cornell University)|Oct 27, 2021

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This survey provides a comprehensive review of self-supervised and few-shot object detection methods, analyzing how self-supervised pretraining improves few-shot generalization by learning robust visual representations from unlabeled data. It highlights state-of-the-art methods like DETReg, which achieves SOTA performance on MS COCO by jointly pretraining the backbone and detector heads via self-supervision.

ABSTRACT

Labeling data is often expensive and time-consuming, especially for tasks such as object detection and instance segmentation, which require dense labeling of the image. While few-shot object detection is about training a model on novel (unseen) object classes with little data, it still requires prior training on many labeled examples of base (seen) classes. On the other hand, self-supervised methods aim at learning representations from unlabeled data which transfer well to downstream tasks such as object detection. Combining few-shot and self-supervised object detection is a promising research direction. In this survey, we review and characterize the most recent approaches on few-shot and self-supervised object detection. Then, we give our main takeaways and discuss future research directions. Project page at https://gabrielhuang.github.io/fsod-survey/

Motivation & Objective

• To systematically review and categorize recent advances in few-shot and self-supervised object detection.
• To analyze the limitations of classification-focused self-supervised representations in object detection and identify improvements.
• To evaluate the performance of self-supervised pretraining on both backbone and detection head components in few-shot settings.
• To identify key trends and future research directions in low-data, semi-supervised, and zero-shot object detection.
• To provide a taxonomy and benchmark comparison of methods across supervised, few-shot, and self-supervised paradigms.

Proposed method

• Categorizes methods based on pretraining strategy: supervised backbone, self-supervised backbone and detection head, or full architecture self-supervision.
• Reviews pretext tasks such as contrastive learning (e.g., SimCLR, MoCo), instance discrimination, and masked autoencoding for self-supervised pretraining.
• Analyzes methods that fine-tune or adapt self-supervised representations to object detection head and region proposal networks (RPN).
• Evaluates performance on standard benchmarks including MS COCO and PASCAL VOC, particularly in low-shot and few-shot regimes.
• Compares self-supervised methods with supervised baselines and examines transferability of representations to detection tasks.
• Introduces and discusses DETReg as a state-of-the-art method that performs self-supervised pretraining on the entire Faster R-CNN architecture.

Experimental results

Research questions

• RQ1 How do self-supervised representations trained for image classification transfer to object detection, and what are their limitations?
• RQ2 What are the key architectural and pretraining components that improve few-shot object detection performance?
• RQ3 How do self-supervised pretraining strategies for the full detector (backbone, RPN, head) compare to those only pretraining the backbone?
• RQ4 What are the performance gains of self-supervised methods in low-data regimes such as mini-COCO or 1% COCO subsets?
• RQ5 How do few-shot, semi-supervised, and zero-shot detection methods relate, and what cross-method insights can be drawn?

Key findings

• Self-supervised pretraining of the entire object detector architecture, including backbone, RPN, and detection head, leads to state-of-the-art performance in few-shot object detection on MS COCO.
• Methods like DETReg achieve SOTA results by jointly pretraining all components via self-supervision, outperforming those that only pretrain the backbone.
• Self-supervised representations pre-trained on ImageNet or COCO with contrastive or instance discrimination objectives show strong transfer to object detection, even when not optimized for localization.
• Performance gains from self-supervised pretraining are especially pronounced in low-data regimes, such as 1% or 5% of MS COCO annotations.
• There is a significant gap between ImageNet Top-1 accuracy and downstream detection performance, indicating that classification-optimized self-supervision is suboptimal for detection.
• Future improvements are likely to come from joint pretraining of detector components and from integrating vision-language priors, as seen in zero-shot detection methods.