[Paper Review] Exploring Category-correlated Feature for Few-shot Image Classification

Jing Xu, Xinglin Pan+3 more|arXiv (Cornell University)|Dec 14, 2021

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper proposes a Category-correlated Feature Corrector (CCF), a generic, backbone-agnostic method that rectifies biased features in few-shot image classification by leveraging category correlations with base classes. By encoding feature similarity to base classes in a latent vector and using a temperature-controlled decoder, CCF generates more representative features closer to class centroids, consistently improving accuracy across benchmarks with diverse backbones and classifiers.

ABSTRACT

Few-shot classification aims to adapt classifiers to novel classes with a few training samples. However, the insufficiency of training data may cause a biased estimation of feature distribution in a certain class. To alleviate this problem, we present a simple yet effective feature rectification method by exploring the category correlation between novel and base classes as the prior knowledge. We explicitly capture such correlation by mapping features into a latent vector with dimension matching the number of base classes, treating it as the logarithm probability of the feature over base classes. Based on this latent vector, the rectified feature is directly constructed by a decoder, which we expect maintaining category-related information while removing other stochastic factors, and consequently being closer to its class centroid. Furthermore, by changing the temperature value in softmax, we can re-balance the feature rectification and reconstruction for better performance. Our method is generic, flexible and agnostic to any feature extractor and classifier, readily to be embedded into existing FSL approaches. Experiments verify that our method is capable of rectifying biased features, especially when the feature is far from the class centroid. The proposed approach consistently obtains considerable performance gains on three widely used benchmarks, evaluated with different backbones and classifiers. The code will be made public.

Motivation & Objective

• To address the bias in feature distribution caused by insufficient training samples in few-shot learning.
• To leverage underutilized knowledge from base classes as prior information for novel class feature rectification.
• To develop a generic, classifier-agnostic method that enhances feature representativeness without modifying existing models.
• To improve decision boundary estimation by generating features closer to class centroids, especially when original features are outliers.

Proposed method

• The CCF is an autoencoder with an encoder that maps a novel image feature into a latent vector matching the number of base classes.
• The latent vector is interpreted as log-probabilities over base classes, encoding category correlation information.
• A base-class classification loss is applied to the latent vector via a temperature-adjusted softmax, encouraging semantic consistency.
• The decoder reconstructs a rectified feature from the latent vector, aiming to preserve category-related information while suppressing noise and outliers.
• The temperature hyperparameter balances feature rectification and reconstruction, with optimal values found at T=0.1 for 1-shot and T=0.02 for 5-shot tasks.
• The method is plug-and-play, compatible with any feature extractor and classifier, and can be integrated into existing few-shot learning frameworks.

Experimental results

Research questions

• RQ1 Can category correlations between novel and base classes serve as effective prior knowledge to reduce feature bias in few-shot learning?
• RQ2 How can we explicitly model and utilize category correlation in a latent space to generate more representative features?
• RQ3 What is the optimal trade-off between feature rectification and reconstruction fidelity in the latent space?
• RQ4 Does the proposed method generalize across different backbones, classifiers, and few-shot learning algorithms?
• RQ5 How does the temperature hyperparameter in the softmax affect the balance between rectification and reconstruction?

Key findings

• The CCF method achieves consistent performance gains across three benchmarks: mini-ImageNet, CUB, and Stanford Cars, with improvements ranging from 2% to 9% on various backbones and classifiers.
• On mini-ImageNet with 1-shot and 5-shot settings, the method achieves 68.88% and 84.59% accuracy respectively, outperforming the baseline by over 3% in 1-shot.
• The ablation study confirms that combining original and rectified features yields the best performance, with the largest gain coming from the CCF component.
• The method generalizes well: it improves performance across different backbones (ResNet-12, Wide ResNet, DenseNet) and classifiers (softmax, cosine, logistic regression).
• Hyperparameter analysis shows that T=0.1 optimizes 1-shot performance, while T=0.02 is optimal for 5-shot, indicating that stronger rectification is needed when data is scarcer.
• Visualization shows that rectified features shift outliers closer to their ground-truth class centers, confirming the method's ability to correct biased feature distributions.