[Paper Review] Few-Shot Learning with Graph Neural Networks

Víctor García, Joan Bruna|arXiv (Cornell University)|Nov 10, 2017

Domain Adaptation and Few-Shot Learning25 references437 citations

TL;DR

The paper casts few-shot learning as supervised interpolation on a fully connected graph built from image collections, and uses a graph neural network with learned edge features to propagate label information.

ABSTRACT

We propose to study the problem of few-shot learning with the prism of inference on a partially observed graphical model, constructed from a collection of input images whose label can be either observed or not. By assimilating generic message-passing inference algorithms with their neural-network counterparts, we define a graph neural network architecture that generalizes several of the recently proposed few-shot learning models. Besides providing improved numerical performance, our framework is easily extended to variants of few-shot learning, such as semi-supervised or active learning, demonstrating the ability of graph-based models to operate well on 'relational' tasks.

Motivation & Objective

• Motivate few-shot learning as inference on a partially observed graphical model constructed from input images.
• Propose a graph neural network (GNN) framework that learns to propagate label information across a set of labeled and unlabeled images.
• Show that the GNN matches or approaches state-of-the-art on Omniglot and Mini-Imagenet with fewer parameters.
• Demonstrate extensions of the framework to semi-supervised and active learning scenarios.

Proposed method

• Represent the image collection as a fully connected graph where nodes are images and edges are learned via a trainable similarity kernel.
• Use a graph neural network with layers that perform message passing, incorporating both node features and learned edge features (Eq. 2 and Eq. 3–4).
• Initialize node features by embedding images with a CNN and concatenating known labels as one-hot vectors; unlabeled nodes use uniform priors over class labels.
• Train end-to-end to predict the label of the query node via a softmax on the final node representations.
• Extend the framework to semi-supervised learning by including unlabeled nodes with uniform label priors, and to active learning by learning which unlabeled node to query (Softmax attention over unlabeled nodes).
• Relate the model to Siamese, Prototypical, and Matching Networks as special cases of graph-based message passing.

Experimental results

Research questions

• RQ1 Can few-shot learning be effectively framed as inference on a graph with learnable edge features and trained end-to-end as a GNN?
• RQ2 Does the GNN approach achieve competitive accuracy on standard few-shot benchmarks with fewer parameters than prior methods?
• RQ3 Can the model naturally extend to semi-supervised and active learning settings without fundamental changes to the training objective?
• RQ4 How does learned edge-feature adjacency learning compare to fixed similarity metrics in propagating label information?
• RQ5 What are the impacts of graph-based message passing depth on performance in dense, fully-connected graphs for few-shot tasks?

Key findings

• The GNN approach achieves competitive accuracy on Omniglot and Mini-Imagenet, matching or surpassing several state-of-the-art methods with substantially fewer parameters.
• The model unifies few-shot, semi-supervised, and active learning within a single graph-based framework.
• Semi-supervised experiments show that unlabeled data can substantially improve performance, sometimes matching higher-label settings with fewer labeled samples.
• Active learning experiments demonstrate the learned querying strategy can outperform random labeling in Mini-Imagenet and Omniglot, improving accuracy.
• The architecture generalizes several existing few-shot models (Siamese, Prototypical, Matching Networks) as special cases of graph-based message passing.
• The method is end-to-end trainable and permutation-invariant with respect to the input set.