[Paper Review] Hierarchical Representation based Query-Specific Prototypical Network for Few-Shot Image Classification

Yaohui Li, Huaxiong Li+2 more|arXiv (Cornell University)|Mar 21, 2021

Domain Adaptation and Few-Shot Learning40 references4 citations

TL;DR

This paper proposes a Query-Specific Prototypical Network (QPN) that enhances few-shot image classification by generating query-specific, region-level prototypes through hierarchical feature representation and attention-based region selection. By achieving both positional and scale-wise semantic alignment, QPN outperforms state-of-the-art methods on five benchmarks, including fine-grained datasets, with a 54.92% accuracy on miniImageNet under 5-way 1-shot settings.

ABSTRACT

Few-shot image classification aims at recognizing unseen categories with a small number of labeled training data. Recent metric-based frameworks tend to represent a support class by a fixed prototype (e.g., the mean of the support category) and make classification according to the similarities between query instances and support prototypes. However, discriminative dominant regions may locate uncertain areas of images and have various scales, which leads to the misaligned metric. Besides, a fixed prototype for one support category cannot fit for all query instances to accurately reflect their distances with this category, which lowers the efficiency of metric. Therefore, query-specific dominant regions in support samples should be extracted for a high-quality metric. To address these problems, we propose a Hierarchical Representation based Query-Specific Prototypical Network (QPN) to tackle the limitations by generating a region-level prototype for each query sample, which achieves both positional and dimensional semantic alignment simultaneously. Extensive experiments conducted on five benchmark datasets (including three fine-grained datasets) show that our proposed method outperforms the current state-of-the-art methods.

Motivation & Objective

• Address the limitation of fixed prototypes in metric-based few-shot learning, which fail to capture query-specific discriminative regions.
• Overcome misalignment in feature space caused by varying object scales and positions across images.
• Improve metric efficiency by filtering out irrelevant background regions and focusing on semantically relevant, discriminative regions.
• Enable robust generalization across domains by learning transferable, query-adaptive representations.
• Replace complex multi-scale modules with a lightweight hierarchical pooling strategy to maintain performance with fewer parameters.

Proposed method

• Employ the Convolutional Block Attention Module (CBAM) to identify discriminative key regions in support and query features.
• Compute local similarity between query regions and support regions to select query-relevant support regions for prototype generation.
• Construct query-specific prototypes by re-weighting selected support regions based on their similarity to the query.
• Implement a hierarchical representation via multi-scale max-pooling to capture scale-invariant features without adding significant parameters.
• Use a learnable generation coefficient (ξ) to balance the influence of different feature scales in the final representation.
• Adopt cosine similarity as the metric function to compute distances between query features and query-specific prototypes.

Experimental results

Research questions

• RQ1 Can query-specific prototypes improve few-shot classification by adapting to the unique spatial and semantic characteristics of each query?
• RQ2 How does hierarchical multi-scale representation enhance robustness to scale variations in discriminative regions?
• RQ3 To what extent does filtering out irrelevant regions through attention and local similarity improve metric accuracy?
• RQ4 Does the proposed method generalize better across domains compared to existing few-shot learning baselines?
• RQ5 Can a lightweight hierarchical pooling mechanism outperform complex multi-scale modules like Inception Operator with fewer parameters?

Key findings

• QPN achieves 54.92% accuracy on miniImageNet under 5-way 1-shot, surpassing the previous SOTA of 54.08% with Gaussian similarity.
• On 5-way 5-shot, QPN reaches 73.18% accuracy, outperforming all compared methods including those with stronger backbones.
• In cross-domain evaluation (miniImageNet → CUB-200-2011), QPN achieves 66.19% accuracy, exceeding the baseline method by 0.62 percentage points.
• QPN uses only 0.150M trainable parameters, making it more parameter-efficient than GCR (1.755M) and comparable to simpler models like ProtoNet.
• The ablation study confirms that the generation coefficient ξ has a moderate influence on performance, with optimal values around 5–7.
• Cosine similarity yields better performance than Gaussian similarity, with a 0.84% improvement on 1-shot and 0.55% on 5-shot settings.