[Paper Review] Few-Shot Learning with Metric-Agnostic Conditional Embeddings

Nathan Hilliard, Lawrence Phillips+4 more|arXiv (Cornell University)|Feb 12, 2018

Digital Imaging for Blood DiseasesComputer Science3 references138 citations

TL;DR

The paper introduces MACO, a metric-agnostic few-shot classifier that conditions class representations on the query image, achieving state-of-the-art on CUB-200 and competitive results on other datasets by learning a flexible, non-metric comparison mechanism.

ABSTRACT

Learning high quality class representations from few examples is a key problem in metric-learning approaches to few-shot learning. To accomplish this, we introduce a novel architecture where class representations are conditioned for each few-shot trial based on a target image. We also deviate from traditional metric-learning approaches by training a network to perform comparisons between classes rather than relying on a static metric comparison. This allows the network to decide what aspects of each class are important for the comparison at hand. We find that this flexible architecture works well in practice, achieving state-of-the-art performance on the Caltech-UCSD birds fine-grained classification task.

Motivation & Objective

• Motivate improving few-shot generalization by moving away from fixed metric spaces toward learned, query-conditioned class representations.
• Propose a four-stage architecture that conditions per-class embeddings on the query image to better capture task-relevant attributes.
• Demonstrate that a learnable classification stage replacing fixed metrics can yield superior performance across fine-grained and general datasets.
• Show that conditioning the class representations on the query image provides a measurable performance boost.

Proposed method

• Four-stage architecture: feature extraction, relational stage, conditioning stage, and a learnable classifier.
• Relational stage uses pairwise comparisons within each class to produce a 128-D class embedding via an averaging scheme.
• Conditioning stage concatenates the class embedding with the query vector and processes it to produce a conditioned 128-D class vector.
• Classification stage uses a learnable, order-agnostic 1D-convolutional network over the set of class vectors followed by a dense softmax layer.
• The conditioning network uses 4 fully connected blocks with batch normalization and ELU activations, with skip connections.
• Relational network g consists of 4 blocks with 128-D hidden layers and a skip connection; outputs are averaged across all pairwise comparisons to yield the class representation.

Experimental results

Research questions

• RQ1 Can a metric-agnostic neural network output be conditioned on the query image to generate more task-relevant class representations in few-shot learning?
• RQ2 Does conditioning improve few-shot performance beyond non-conditioned metric-agnostic designs?
• RQ3 How does the MACO approach compare to established baselines (Matching Networks, Prototypical Networks, Meta-LSTM, MAML) across fine-grained and broader datasets?

Key findings

• MACO achieves state-of-the-art 1-shot and 5-shot accuracy on Caltech-UCSD Birds (CUB-200): 60.76% (1-shot) and 74.96% (5-shot).
• Removing conditioning (MA w/o cond.) reduces performance to 55.86% (1-shot) and 69.49% (5-shot) on CUB-200, indicating conditioning contributes roughly 5 percentage points.
• On mini ImageNet, MACO reaches 41.09% (1-shot) and 58.32% (5-shot), competitive with several baselines but trailing top meta-learning methods on 1-shot.
• On mini DogsNet, MACO achieves 39.10% (1-shot) and 54.45% (5-shot), ranking between matching networks and MAML depending on shot."
• The architecture demonstrates that a learnable classifier with query-conditioned class representations can outperform traditional metric-based approaches on fine-grained datasets and remain competitive on broader datasets.