[Paper Review] Few-Shot Learning by Integrating Spatial and Frequency Representation

Xiangyu Chen, Guanghui Wang|arXiv (Cornell University)|May 11, 2021

Domain Adaptation and Few-Shot Learning43 references4 citations

TL;DR

This paper proposes integrating spatial and frequency domain representations via Discrete Cosine Transform (DCT) preprocessing to improve few-shot image classification. By fusing features from both domains using a CNN backbone, the method achieves significant accuracy gains—up to 62.30% on 5-way 1-shot miniImageNet—demonstrating that frequency information complements spatial features and boosts model discrimination.

ABSTRACT

Human beings can recognize new objects with only a few labeled examples, however, few-shot learning remains a challenging problem for machine learning systems. Most previous algorithms in few-shot learning only utilize spatial information of the images. In this paper, we propose to integrate the frequency information into the learning model to boost the discrimination ability of the system. We employ Discrete Cosine Transformation (DCT) to generate the frequency representation, then, integrate the features from both the spatial domain and frequency domain for classification. The proposed strategy and its effectiveness are validated with different backbones, datasets, and algorithms. Extensive experiments demonstrate that the frequency information is complementary to the spatial representations in few-shot classification. The classification accuracy is boosted significantly by integrating features from both the spatial and frequency domains in different few-shot learning tasks.

Motivation & Objective

• Address the challenge of few-shot image classification with limited labeled examples, especially in data-scarce domains like medical imaging.
• Overcome the limitation of existing few-shot learning models that rely solely on spatial image features.
• Investigate whether frequency domain representations, derived via DCT, can enhance feature discrimination and model generalization.
• Demonstrate the effectiveness of fusing spatial and frequency features across diverse backbones, datasets, and few-shot learning protocols.

Proposed method

• Apply Discrete Cosine Transform (DCT) to input images to generate frequency-domain representations before feeding them into a CNN backbone.
• Use two parallel CNN branches: one processing original spatial-domain images (CNN(s)), and another processing DCT-transformed frequency-domain features (CNN(f)).
• Concatenate the feature embeddings from both branches (spatial and frequency) to form a unified representation for classification.
• Employ static channel selection in the DCT domain, focusing on low-frequency components to reduce computational cost while preserving discriminative information.
• Integrate the proposed DCT-based frequency module as a plug-in component into existing few-shot learning frameworks such as baseline++.
• Utilize t-SNE visualization to compare feature clustering quality between models using only spatial features versus fused spatial-frequency features.

Experimental results

Research questions

• RQ1 Can frequency domain representations, extracted via DCT, improve few-shot classification performance when combined with spatial features?
• RQ2 How does the size of the DCT filter affect classification accuracy in few-shot learning tasks?
• RQ3 Does the integration of spatial and frequency features lead to better feature clustering and generalization compared to using spatial features alone?
• RQ4 How robust is the proposed method across different backbone architectures (e.g., ResNet10, ResNet18, ResNet34) and few-shot learning benchmarks?
• RQ5 What is the relative contribution of data augmentation versus frequency domain integration in improving model accuracy?

Key findings

• The proposed method, which fuses spatial and frequency features, achieves a 62.30% top-1 accuracy on the 5-way 1-shot miniImageNet benchmark using ResNet10, outperforming the baseline++(s) model by 4.78 percentage points.
• For the 5-way 5-shot task, the fused model reaches 79.93% accuracy on miniImageNet with ResNet10, a 4.37% improvement over the spatial-only baseline.
• The frequency-only branch (CNN(f)) consistently outperforms the spatial-only baseline across all backbones and tasks, indicating that frequency features alone provide meaningful discrimination.
• The integration of both domains yields the highest performance gains, with improvements ranging from 4.37% to 5.75% over the spatial-only baseline across different backbones.
• The DCT filter size has minimal impact on performance when using static channel selection, suggesting that small filter sizes (e.g., 8×8) are sufficient for effective feature extraction.
• t-SNE visualization confirms that the fused model produces more compact and well-separated feature clusters, indicating enhanced feature discrimination and generalization capability.