[Paper Review] Centroid Networks for Few-Shot Clustering and Unsupervised Few-Shot Classification.

Gabriel Huang, Hugo Larochelle+1 more|arXiv (Cornell University)|Feb 22, 2019

Domain Adaptation and Few-Shot Learning13 citations

TL;DR

This paper introduces Centroid Networks, a few-shot clustering method that learns task-specific representations to improve clustering performance without labels. By framing clustering as a meta-learning problem and introducing unsupervised few-shot classification, it achieves performance competitive with supervised few-shot classification methods on Omniglot and miniImageNet, despite using no support set labels.

ABSTRACT

Traditional clustering algorithms such as K-means rely heavily on the nature of the chosen metric or data representation. To get meaningful clusters, these representations need to be tailored to the downstream task (e.g. cluster photos by object category, cluster faces by identity). Therefore, we frame clustering as a meta-learning task, few-shot clustering, which allows us to specify how to cluster the data at the meta-training level, despite the clustering algorithm itself being unsupervised. We propose Centroid Networks, a simple and efficient few-shot clustering method based on learning representations which are tailored both to the task to solve and to its internal clustering module. We also introduce unsupervised few-shot classification, which is conceptually similar to few-shot clustering, but is strictly harder than supervised* few-shot classification and therefore allows direct comparison with existing supervised few-shot classification methods. On Omniglot and miniImageNet, our method achieves accuracy competitive with popular supervised few-shot classification algorithms, despite using *no labels* from the support set. We also show performance competitive with state-of-the-art learning-to-cluster methods.

Motivation & Objective

• To address the limitation of traditional clustering methods that rely heavily on pre-defined metrics or representations not aligned with downstream tasks.
• To frame clustering as a meta-learning problem—few-shot clustering—where the clustering strategy is learned during meta-training.
• To introduce unsupervised few-shot classification as a harder, more direct benchmark for comparing with supervised few-shot classification methods.
• To develop a representation learning method that tailors features to both the clustering task and its internal module, improving cluster quality.

Proposed method

• Proposes Centroid Networks, a neural network architecture that learns representations optimized for a specific clustering task and its internal clustering module.
• Uses meta-learning to train the model so that it generalizes to new few-shot clustering tasks during inference, even without labels in the support set.
• Introduces an unsupervised few-shot classification setting where the model must classify support sets into clusters without any supervision, making it strictly harder than supervised variants.
• Leverages a centroid-based clustering module where each class is represented by a prototype (centroid), and samples are assigned based on distance to these centroids.
• Trains the model end-to-end using a contrastive loss that encourages intra-cluster compactness and inter-cluster separation in the embedding space.
• Applies the learned representations to both few-shot clustering and unsupervised few-shot classification, enabling zero-shot generalization to novel classes.

Experimental results

Research questions

• RQ1 Can clustering be effectively framed as a few-shot meta-learning problem to improve generalization to novel tasks?
• RQ2 Can a representation learning method be designed to be both task-specific and compatible with an internal clustering module?
• RQ3 Does unsupervised few-shot classification provide a meaningful benchmark for comparing with supervised few-shot classification?
• RQ4 Can a method achieve competitive performance on few-shot clustering and classification without using any labels from the support set?

Key findings

• Centroid Networks achieve performance competitive with state-of-the-art supervised few-shot classification methods on Omniglot and miniImageNet, despite using no labels from the support set.
• The proposed unsupervised few-shot classification setting provides a meaningful and strictly harder benchmark than supervised variants, enabling direct comparison with existing methods.
• The method outperforms existing learning-to-cluster approaches on both Omniglot and miniImageNet, demonstrating strong generalization to novel classes.
• The use of task-aware representation learning significantly improves clustering quality compared to standard clustering baselines like K-means with standard feature extractors.