[Paper Review] Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector

Qi Fan, Wei Zhuo+2 more|arXiv (Cornell University)|Aug 6, 2019

Advanced Neural Network Applications63 references37 citations

TL;DR

The paper introduces a few-shot object detection network with Attention-RPN and Multi-Relation Detector trained via contrastive learning, plus a 1000-category FSOD dataset, achieving state-of-the-art results without fine-tuning.

ABSTRACT

Conventional methods for object detection typically require a substantial amount of training data and preparing such high-quality training data is very labor-intensive. In this paper, we propose a novel few-shot object detection network that aims at detecting objects of unseen categories with only a few annotated examples. Central to our method are our Attention-RPN, Multi-Relation Detector and Contrastive Training strategy, which exploit the similarity between the few shot support set and query set to detect novel objects while suppressing false detection in the background. To train our network, we contribute a new dataset that contains 1000 categories of various objects with high-quality annotations. To the best of our knowledge, this is one of the first datasets specifically designed for few-shot object detection. Once our few-shot network is trained, it can detect objects of unseen categories without further training or fine-tuning. Our method is general and has a wide range of potential applications. We produce a new state-of-the-art performance on different datasets in the few-shot setting. The dataset link is https://github.com/fanq15/Few-Shot-Object-Detection-Dataset.

Motivation & Objective

• Develop a general few-shot object detection model that detects unseen categories without retraining.
• Leverage support-query similarity to enhance proposal generation and detection accuracy.
• Introduce an attention mechanism in RPN and a multi-relation detector to suppress background false positives.
• Construct a large, diverse FSOD dataset with 1000 categories for robust few-shot evaluation.
• Demonstrate strong performance on ImageNet Detection and MS COCO without fine-tuning.

Proposed method

• Use a weight-shared Faster R-CNN framework with an Attention-RPN that incorporates support information via depth-wise cross-correlation to filter proposals.
• Integrate a Multi-Relation Detector with global, local, and patch heads to measure support-query relationships and re-score proposals.
• Apply a 2-way contrastive training strategy that trains on same-category matches and different-category distinctions to improve discrimination.
• Train end-to-end on FSOD with optional pretraining on ImageNet and COCO, using a 2-way 5-shot contrastive scheme as the full model.
• Fuse support features by averaging across same-category supports to form robust support representations for the attention RPN and detector.

Experimental results

Research questions

• RQ1 Can a shared-weight detection framework be guided by support-image information to detect novel categories without retraining?
• RQ2 Does an attention-based RPN improve proposal quality for few-shot detection compared to a vanilla RPN?
• RQ3 Do multiple relation heads (global, local, patch) in a detector provide complementary gains for few-shot matching?
• RQ4 Does a 2-way contrastive training strategy that distinguishes different categories improve few-shot detection performance?
• RQ5 How does the proposed FSOD dataset affect generalization to unseen categories on standard benchmarks?

Key findings

• Attention-RPN improves recall and ABO score compared to regular RPN, yielding higher AP50 and AP75.
• Full multi-relation detector with all three heads achieves best AP50 and AP75, surpassing single-head configurations.
• 2-way 5-shot contrastive training yields strong gains over naive training, achieving state-of-the-art results without fine-tuning on several datasets.
• Models trained on FSOD generalize better to unseen categories than those trained on COCO, especially in 5-shot settings.
• The FSOD dataset enables superior few-shot performance and diversity benefits over traditional datasets with fewer categories.