[Paper Review] Adaptive Distribution Calibration for Few-Shot Learning with Hierarchical Optimal Transport

Dandan Guo, Long Tian+3 more|arXiv (Cornell University)|Oct 9, 2022

Domain Adaptation and Few-Shot Learning12 citations

TL;DR

This paper proposes a hierarchical optimal transport (H-OT) framework for few-shot learning that adaptively calibrates feature distributions by learning a transport plan between novel samples and base classes. By modeling base classes as distributions over their samples via low-level optimal transport and using high-level OT to compute adaptive weights, the method improves few-shot generalization without fine-tuning, outperforming baselines like Free-Lunch even with fewer generated features.

ABSTRACT

Few-shot classification aims to learn a classifier to recognize unseen classes during training, where the learned model can easily become over-fitted based on the biased distribution formed by only a few training examples. A recent solution to this problem is calibrating the distribution of these few sample classes by transferring statistics from the base classes with sufficient examples, where how to decide the transfer weights from base classes to novel classes is the key. However, principled approaches for learning the transfer weights have not been carefully studied. To this end, we propose a novel distribution calibration method by learning the adaptive weight matrix between novel samples and base classes, which is built upon a hierarchical Optimal Transport (H-OT) framework. By minimizing the high-level OT distance between novel samples and base classes, we can view the learned transport plan as the adaptive weight information for transferring the statistics of base classes. The learning of the cost function between a base class and novel class in the high-level OT leads to the introduction of the low-level OT, which considers the weights of all the data samples in the base class. Experimental results on standard benchmarks demonstrate that our proposed plug-and-play model outperforms competing approaches and owns desired cross-domain generalization ability, indicating the effectiveness of the learned adaptive weights.

Motivation & Objective

• Address the limitation of heuristic or non-adaptive transfer weights in distribution calibration for few-shot learning.
• Improve generalization by learning principled, data-driven similarity between novel samples and base classes.
• Enable effective cross-domain few-shot learning by transferring statistics from source to target domains.
• Develop a plug-and-play framework compatible with any pre-trained feature extractor, avoiding costly fine-tuning.
• Formulate a two-level optimal transport system to model both intra-class sample weights and inter-class similarity in a unified, differentiable manner.

Proposed method

• Formulate the distribution calibration task as a high-level optimal transport (H-OT) problem between a novel sample distribution and a base class distribution.
• Introduce a low-level optimal transport problem to compute the cost between a novel sample and a base class by modeling the base class as a distribution over its individual samples.
• Use the solution from the low-level OT (i.e., sample weights within a base class) as the cost function for the high-level OT, enabling adaptive, data-driven similarity learning.
• Learn an optimal transport plan from the high-level OT that serves as an adaptive weight matrix, assigning different contributions to each base class for each novel sample.
• Construct a plug-and-play framework that operates at the feature level using pre-trained backbones, avoiding fine-tuning of the feature extractor.
• Apply the learned transport plan to transfer statistics (e.g., mean features) from base classes to novel samples for improved classification.

Experimental results

Research questions

• RQ1 Can a hierarchical optimal transport framework learn more effective and adaptive transfer weights between base and novel classes than heuristic or fixed-distance methods?
• RQ2 Does modeling base classes as distributions over their samples (via low-level OT) improve the representation of class-level similarity compared to using unweighted averages?
• RQ3 Can the proposed H-OT method generalize across domains, particularly in cross-domain few-shot learning settings?
• RQ4 How does the adaptive weight matrix learned via H-OT compare to top-k selection (e.g., k=2) in terms of sample efficiency and accuracy?
• RQ5 To what extent does the transport plan reflect semantic relationships between base and novel classes?

Key findings

• H-OT outperforms the Free-Lunch baseline on miniImageNet across multiple backbones (Conv4, Conv6, ResNet10, ResNet18), demonstrating consistent gains in 5-way-1-shot accuracy.
• H-OT achieves ~68.5% accuracy with only 50 generated features, while Free-Lunch requires 750 generated features to reach a similar performance level.
• The learned transport plan successfully identifies semantically relevant base classes (e.g., 'elephant', 'bear' for 'plain'), indicating that the method captures meaningful semantic correlations.
• Even with fixed cost functions, H-OT outperforms Free-Lunch, proving the value of the adaptive weight matrix learned via high-level OT.
• The method maintains strong performance in cross-domain few-shot learning, showing robustness and generalization beyond the source domain.
• The framework is agnostic to backbone architecture and does not require fine-tuning, enabling plug-and-play integration with any pre-trained feature extractor.