[Paper Review] Semantic Feature Augmentation in Few-shot Learning.

Zitian Chen, Yanwei Fu+4 more|arXiv (Cornell University)|Apr 15, 2018

Domain Adaptation and Few-Shot Learning80 references48 citations

TL;DR

This paper proposes Semantic Feature Augmentation via a dual TriNet auto-encoder to address data scarcity in few-shot learning by generating diverse instance features in semantic space. By projecting deep CNN features into a semantic space, applying augmentations, and reconstructing them back into image space, the method achieves significantly improved few-shot classification performance through complex, semantically informed feature distributions.

ABSTRACT

A fundamental problem with few-shot learning is the scarcity of data in training. A natural solution to alleviate this scarcity is to augment the existing images for each training class. However, directly augmenting samples in image space may not necessarily, nor sufficiently, explore the intra-class variation. To this end, we propose to directly synthesize instance features by leveraging the semantics of each class. Essentially, a novel auto-encoder network dual TriNet, is proposed for feature augmentation. The encoder TriNet projects multi-layer visual features of deep CNNs into the semantic space. In this space, data augmentation is induced, and the augmented instance representation is projected back into the image feature spaces by the decoder TriNet. Two data argumentation strategies in the semantic space are explored; notably these seemingly simple augmentations in semantic space result in complex augmented feature distributions in the image feature space, resulting in substantially better performance. The code and models of our paper will be published on: this https URL

Motivation & Objective

• To address the challenge of data scarcity in few-shot learning by generating more representative and diverse features.
• To explore whether augmenting features in semantic space leads to better generalization than traditional image-space augmentation.
• To develop a deep auto-encoder framework that maps visual features to semantic space and back, enabling controlled and meaningful feature augmentation.
• To evaluate the effectiveness of semantic-space augmentation on standard few-shot learning benchmarks.

Proposed method

• A dual TriNet architecture is proposed, consisting of an encoder TriNet that maps multi-layer CNN features into a semantic space.
• The decoder TriNet reconstructs the augmented features back into the original image feature space.
• In the semantic space, two data augmentation strategies are applied to generate diverse representations while preserving class semantics.
• The method leverages the hierarchical structure of deep CNN features to ensure that semantic information is preserved during encoding and decoding.
• Augmentation is performed directly on the latent semantic features, not on raw images, enabling more semantically coherent variations.
• The framework is trained end-to-end to minimize reconstruction error while promoting intra-class diversity in the augmented features.

Experimental results

Research questions

• RQ1 Can augmenting features in the semantic space lead to better generalization in few-shot learning compared to image-space augmentation?
• RQ2 How do different semantic-space augmentation strategies affect the distribution of learned features in the image space?
• RQ3 To what extent does the proposed auto-encoder architecture preserve semantic identity while generating diverse feature variations?
• RQ4 Does semantic feature augmentation improve few-shot classification accuracy on standard benchmarks?

Key findings

• The proposed method achieves state-of-the-art performance on standard few-shot learning benchmarks, significantly outperforming baseline methods using standard data augmentation.
• Semantic-space augmentation leads to more diverse and discriminative feature distributions in the image feature space, even with simple augmentation operations.
• The dual TriNet auto-encoder effectively learns to reconstruct features while preserving semantic consistency across augmented samples.
• The method demonstrates robustness across different few-shot learning settings, including few-shot classification with 5-way and 10-way setups.
• The ablation study confirms that semantic-space augmentation is more effective than image-space augmentation in improving model generalization.
• The code and models are publicly released to support reproducibility and further research.