[Paper Review] Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation

Hung-Yu Tseng, Hsin-Ying Lee+2 more|arXiv (Cornell University)|Jan 23, 2020

Domain Adaptation and Few-Shot Learning65 references179 citations

TL;DR

The paper introduces feature-wise transformation layers to augment feature activations in metric-based few-shot classifiers and uses a learning-to-learn strategy to optimize these layers for better generalization to unseen domains under domain shift.

ABSTRACT

Few-shot classification aims to recognize novel categories with only few labeled images in each class. Existing metric-based few-shot classification algorithms predict categories by comparing the feature embeddings of query images with those from a few labeled images (support examples) using a learned metric function. While promising performance has been demonstrated, these methods often fail to generalize to unseen domains due to large discrepancy of the feature distribution across domains. In this work, we address the problem of few-shot classification under domain shifts for metric-based methods. Our core idea is to use feature-wise transformation layers for augmenting the image features using affine transforms to simulate various feature distributions under different domains in the training stage. To capture variations of the feature distributions under different domains, we further apply a learning-to-learn approach to search for the hyper-parameters of the feature-wise transformation layers. We conduct extensive experiments and ablation studies under the domain generalization setting using five few-shot classification datasets: mini-ImageNet, CUB, Cars, Places, and Plantae. Experimental results demonstrate that the proposed feature-wise transformation layer is applicable to various metric-based models, and provides consistent improvements on the few-shot classification performance under domain shift.

Motivation & Objective

• Address domain shift in metric-based few-shot classification by augmenting feature distributions during training.
• Propose feature-wise transformation layers that apply affine transforms to intermediate features.
• Develop a learning-to-learn algorithm to optimize the hyper-parameters of these transformation layers.
• Demonstrate improved generalization across multiple domain shifts on standard few-shot benchmarks.

Proposed method

• Insert feature-wise transformation layers after batch normalization in the feature encoder to apply affine transformations with gamma and beta sampled from Gaussian distributions parameterized by learned hyper-parameters.
• Formally, gamma ~ N(1, softplus(theta_gamma)) and beta ~ N(0, softplus(theta_beta)), with z_hat = gamma * z + beta where z is intermediate activation.
• Use multiple layers across the encoder to simulate diverse feature distributions across domains.
• Optimize hyper-parameters theta_gamma and theta_beta with a learning-to-learn loop that trains on pseudo-seen and pseudo-unseen domains to maximize generalization.
• Jointly optimize the metric-based model parameters and the feature-wise transformation hyper-parameters via a bilevel-like scheme (Eq. 5–7).
• Evaluate with three metric-based backbones (MatchingNet, RelationNet, GNN) on 5-way tasks across five datasets (mini-ImageNet, CUB, Cars, Places, Plantae) using a ResNet-10 backbone.

Experimental results

Research questions

• RQ1 Can feature-wise transformation layers improve domain generalization for few-shot classification?
• RQ2 How can hyper-parameters of these layers be optimized to capture cross-domain feature distribution variations?
• RQ3 Do learning-to-learn strategies effectively tune augmentation parameters to enhance performance on unseen domains?
• RQ4 How do the augmented representations affect several metric-based baselines (MatchingNet, RelationNet, GNN) across diverse domains?

Key findings

• Feature-wise transformation layers yield consistent improvements in few-shot accuracy under domain shift across all tested backbones.
• The learning-to-learn optimization of transformation parameters further improves generalization beyond manually tuned values.
• t-SNE visualizations show reduced domain distance when using the transformation layers, indicating better cross-domain alignment of features.
• Learning-to-learn optimization outperforms fixed, pre-determined parameter settings in leave-one-domain-out evaluations.
• GNN with the proposed layers achieves strong performance on both seen and unseen domains, often surpassing baselines.