[Paper Review] Variational Feature Disentangling for Fine-Grained Few-Shot Classification

Jingyi Xu, Hieu Lê+3 more|arXiv (Cornell University)|Oct 7, 2020

Domain Adaptation and Few-Shot Learning45 references4 citations

TL;DR

This paper proposes a variational feature disentanglement framework for fine-grained few-shot learning that separates features into class-discriminative and intra-class variance components. By modeling intra-class variations via a shared isotropic Gaussian distribution learned from base classes, the method generates diverse, realistic augmentations that preserve class identity, achieving state-of-the-art performance on CUB, NAB, and Stanford Dogs datasets with gains up to 4.6% over prior methods.

ABSTRACT

Fine-grained few-shot recognition often suffers from the problem of training data scarcity for novel categories.The network tends to overfit and does not generalize well to unseen classes due to insufficient training data. Many methods have been proposed to synthesize additional data to support the training. In this paper, we focus one enlarging the intra-class variance of the unseen class to improve few-shot classification performance. We assume that the distribution of intra-class variance generalizes across the base class and the novel class. Thus, the intra-class variance of the base set can be transferred to the novel set for feature augmentation. Specifically, we first model the distribution of intra-class variance on the base set via variational inference. Then the learned distribution is transferred to the novel set to generate additional features, which are used together with the original ones to train a classifier. Experimental results show a significant boost over the state-of-the-art methods on the challenging fine-grained few-shot image classification benchmarks.

Motivation & Objective

• Address the challenge of data scarcity in fine-grained few-shot classification, where limited training samples lead to overfitting and poor generalization.
• Overcome limitations of existing data augmentation methods that suffer from mode collapse or introduce irrelevant variations.
• Ensure augmented samples maintain class-discriminative features while exhibiting high intra-class diversity.
• Develop a method that generalizes well from base classes to novel classes by modeling shared intra-class variations across categories.
• Improve few-shot classification accuracy by disentangling features into discriminative and variance components, enabling effective, distribution-based augmentation.

Proposed method

• Disentangle image features into two components: a class-discriminative feature $ z_I $ obtained via max-pooling the feature map, and an intra-class variance feature $ z_V $ modeled via variational inference.
• Use a Variational Autoencoder (VAE) to learn a shared isotropic Gaussian distribution for intra-class variations across all classes, trained on base classes with abundant data.
• Sample multiple times from the learned $ z_V $ distribution to generate diverse intra-class variations, which are added to the class-discriminative features $ z_I $ to form augmented features.
• Train the model using two supervision signals: a cross-entropy classification loss on $ z_I $ and a VAE reconstruction loss on $ z_V $, ensuring disentanglement.
• Apply the trained framework to augment few-shot support sets by generating new features for novel classes using the same distribution, preserving class identity while increasing diversity.
• Use t-SNE visualization and clustering metrics (DBI, intra/inter-class distances) to evaluate the realism and distributional fidelity of generated features.

Experimental results

Research questions

• RQ1 Can a shared, disentangled representation of intra-class variations improve data augmentation in fine-grained few-shot learning?
• RQ2 Does modeling intra-class variance via a variational autoencoder with a common isotropic Gaussian distribution lead to more realistic and diverse augmented samples?
• RQ3 Can such a disentangled feature augmentation scheme preserve class-discriminative features while increasing intra-class variance?
• RQ4 How does the distribution of generated features compare to real data in terms of cluster tightness and inter-class separation?
• RQ5 To what extent does this method outperform existing data augmentation baselines in few-shot fine-grained classification?

Key findings

• The proposed method achieves state-of-the-art performance on the CUB dataset, with 91.48% accuracy in the 5-shot setting, outperforming the previous SOTA method by 4.6 percentage points.
• On the NAB dataset, the method achieves 95.22% accuracy in the 5-shot setting, a 1.04% improvement over the previous SOTA.
• The generated features from our method closely approximate the distribution of real, unseen features, with a Davies–Bouldin index (DBI) of 2.53, close to the real data's 3.02 and significantly better than competing methods.
• The intra-class variance of augmented features (27.27) is much closer to real data (28.97) than other methods, while the $Δ$-encoder method reduces variance to 10.34, indicating distribution collapse.
• t-SNE visualization confirms that our generated samples lie near real data points of the same class, whereas the $Δ$-encoder creates new, isolated clusters that deviate from the real data distribution.
• The method maintains high performance even with increasing numbers of augmented samples, indicating that the generated features are meaningful and non-redundant.