[Paper Review] Task Discrepancy Maximization for Fine-grained Few-Shot Classification

SuBeen Lee, WonJun Moon+1 more|arXiv (Cornell University)|Jul 4, 2022

Domain Adaptation and Few-Shot Learning8 citations

TL;DR

This paper proposes Task Discrepancy Maximization (TDM), a novel module for fine-grained few-shot classification that enhances feature discrimination by learning class-specific channel weights. TDM uses a Support Attention Module (SAM) to identify discriminative channels from support sets and a Query Attention Module (QAM) to adaptively emphasize object-relevant channels in query images, significantly improving accuracy across CUB, Aircraft, and meta-iNat benchmarks.

ABSTRACT

Recognizing discriminative details such as eyes and beaks is important for distinguishing fine-grained classes since they have similar overall appearances. In this regard, we introduce Task Discrepancy Maximization (TDM), a simple module for fine-grained few-shot classification. Our objective is to localize the class-wise discriminative regions by highlighting channels encoding distinct information of the class. Specifically, TDM learns task-specific channel weights based on two novel components: Support Attention Module (SAM) and Query Attention Module (QAM). SAM produces a support weight to represent channel-wise discriminative power for each class. Still, since the SAM is basically only based on the labeled support sets, it can be vulnerable to bias toward such support set. Therefore, we propose QAM which complements SAM by yielding a query weight that grants more weight to object-relevant channels for a given query image. By combining these two weights, a class-wise task-specific channel weight is defined. The weights are then applied to produce task-adaptive feature maps more focusing on the discriminative details. Our experiments validate the effectiveness of TDM and its complementary benefits with prior methods in fine-grained few-shot classification.

Motivation & Objective

• To address the challenge of distinguishing fine-grained classes with similar overall appearances by focusing on discriminative local details.
• To overcome the limitations of existing few-shot methods that treat all feature channels equally, which harms performance on fine-grained datasets.
• To develop a task-adaptive feature alignment mechanism that dynamically weights channels based on class-specific discriminative power.
• To ensure robustness against bias from support sets by incorporating query-aware attention for dynamic adaptation.

Proposed method

• TDM introduces a Support Attention Module (SAM) that computes class-wise channel weights based on support set features, highlighting channels with high discriminative power.
• It employs a Query Attention Module (QAM) that generates query-specific channel weights to emphasize object-relevant features, reducing noise and improving localization.
• The method computes channel weights by measuring similarity between individual channel maps and the spatially averaged feature map, which encodes object-level semantics.
• TDM combines SAM and QAM weights to produce a task-specific, class-wise channel weight that reweights feature maps for improved discrimination.
• The approach is compatible with standard metrics like Euclidean and cosine distance, enabling integration with existing few-shot learning frameworks.
• Average pooling is used to compute the reference spatially-averaged feature map, as it is more robust to noise than max-pooling.

Figure 1 : Effect of the channel weight in the CUB dataset. (a) Existing methods treat channels of feature maps equally. In such case, high variance channels within a class highly likely to disturb the classification task, where channel variance represents the channel-wise variances of feature maps

Experimental results

Research questions

• RQ1 Can learning class-specific channel weights improve few-shot classification performance on fine-grained datasets where overall appearance is highly similar?
• RQ2 How can a module dynamically adapt channel importance based on both support and query features to reduce bias and improve localization?
• RQ3 To what extent does TDM outperform existing feature alignment methods in fine-grained few-shot learning benchmarks?
• RQ4 Is the proposed method robust and compatible across different distance metrics and backbone architectures?

Key findings

• TDM achieves a 69.94% accuracy on the CUB dataset with a 1-shot setting, outperforming the baseline ProtoNet by 7.04 percentage points.
• On the Aircraft dataset, TDM improves 1-shot accuracy from 47.37% to 50.55% and 5-shot accuracy from 68.96% to 71.12%.
• When combined with FRN, TDM boosts 1-shot accuracy on meta-iNat from 61.98% to 63.97% and 5-shot accuracy from 80.04% to 81.60%.
• Ablation studies confirm that both SAM and QAM are effective and complementary, with SAM providing a 11% gain and QAM adding consistent improvement.
• TDM maintains strong performance with cosine distance, showing compatibility with multiple metric learning frameworks.
• The average pooling function outperforms max-pooling, confirming its robustness to noise in feature representation.

Figure 2 : Visualization of pooling results. Each column shows the locations where each pooling method focuses on the image. The second and third columns visualize the results of average pooling and the max pooling, respectively. GAP tends to concentrate on the object parts in the images, while GMP