[Paper Review] Dynamic Input Structure and Network Assembly for Few-Shot Learning

Nathan Hilliard, Nathan O. Hodas+1 more|arXiv (Cornell University)|Aug 22, 2017

Domain Adaptation and Few-Shot Learning7 references3 citations

TL;DR

This paper proposes a dynamic input structure and network assembly technique for few-shot learning that enables a single model to generalize across varying numbers of support examples per class, using shared weights in a statically compiled graph (e.g., TensorFlow). By training on batch-wise varying example sizes, the model achieves significantly higher accuracy—up to 90.3% on Caltech-UCSD Birds—compared to fixed-size baselines, demonstrating robust generalization without retraining.

ABSTRACT

The ability to learn from a small number of examples has been a difficult problem in machine learning since its inception. While methods have succeeded with large amounts of training data, research has been underway in how to accomplish similar performance with fewer examples, known as one-shot or more generally few-shot learning. This technique has been shown to have promising performance, but in practice requires fixed-size inputs making it impractical for production systems where class sizes can vary. This impedes training and the final utility of few-shot learning systems. This paper describes an approach to constructing and training a network that can handle arbitrary example sizes dynamically as the system is used.

Motivation & Objective

• To address the limitation of few-shot learning models requiring fixed-size inputs, which hinders deployment in real-world systems with variable class sizes.
• To enable a single deep learning model to handle arbitrary example counts per class at inference time, improving practicality for production systems.
• To reduce overfitting and improve generalization by training the model on diverse example sizes during optimization.
• To maintain high performance across different shot settings (e.g., 2-shot to 5-shot) without retraining or architectural changes.

Proposed method

• The model uses a siamese two-stage architecture: a relational stage that computes class embeddings via pairwise comparisons of support examples, followed by a metric learning stage that compares the query to each class embedding.
• Image features are extracted using a pre-trained ResNet-50 (or simpler CNN for Omniglot), reducing dimensionality to 2048-d vectors before processing.
• The relational stage computes a class embedding as the average of all pairwise representations: $ R(c_n) = \frac{1}{\binom{n}{2}} \sum_{i<j} g_\theta(c_i, c_j) $, where $ g_\theta $ is a shared Siamese network.
• Dynamic network assembly is achieved by precomputing input-output tensor mappings for different example sizes, enabling batch-wise size-agnostic inference via an in-memory lookup table.
• The model is trained end-to-end using stochastic gradient descent with momentum, with each training batch containing randomly sampled example sizes (e.g., 2 to 5 shots) to promote generalization.
• Shared weights across all example sizes allow parameter efficiency and enable the model to generalize across shot settings without architectural modification.

Experimental results

Research questions

• RQ1 Can a few-shot learning model generalize effectively across varying numbers of support examples per class without retraining?
• RQ2 Does training on dynamically varying example sizes reduce overfitting and improve generalization to unseen shot settings?
• RQ3 Can a statically compiled deep learning framework (e.g., TensorFlow) support dynamic input sizes through weight sharing and runtime assembly?
• RQ4 How does performance on a fixed-size model compare to a dynamically assembled model when evaluated on unseen example counts?

Key findings

• The dynamic input model achieved 90.3% accuracy on the 5-shot Caltech-UCSD Birds benchmark, significantly outperforming the best fixed-size model (74.7% on 5-shot).
• Even when evaluated on higher shot settings than it was trained on, the dynamic model maintained high performance, indicating strong generalization across shot levels.
• The dynamic model outperformed all fixed-size baselines across all shot settings on both Caltech-UCSD Birds and Omniglot, with gains of up to 15.5 percentage points on 5-shot Omniglot.
• Randomly varying example sizes during training reduced overfitting and improved robustness, as evidenced by consistent performance across shot levels.
• The use of pre-trained features and shared weights enabled high performance with minimal architectural changes and low overhead.
• The dynamic network assembly technique successfully enabled size-agnostic inference in a static graph framework, making the model production-ready.