[Paper Review] One-Class Meta-Learning: Towards Generalizable Few-Shot Open-Set Classification

Jedrzej Kozerawski, Matthew Turk|arXiv (Cornell University)|Sep 14, 2021

Domain Adaptation and Few-Shot Learning4 citations

TL;DR

This paper proposes two novel meta-learning methods—Meta-BCE and OCML—for few-shot one-class and multiclass open-set classification, enabling any existing few-shot learning model to detect unknown classes without retraining or performance degradation. The methods achieve state-of-the-art results on miniImageNet, tieredImageNet, and CUB-200-2011, outperforming prior approaches in both open-set and closed-set settings, especially in low-shot regimes.

ABSTRACT

Real-world classification tasks are frequently required to work in an open-set setting. This is especially challenging for few-shot learning problems due to the small sample size for each known category, which prevents existing open-set methods from working effectively; however, most multiclass few-shot methods are limited to closed-set scenarios. In this work, we address the problem of few-shot open-set classification by first proposing methods for few-shot one-class classification and then extending them to few-shot multiclass open-set classification. We introduce two independent few-shot one-class classification methods: Meta Binary Cross-Entropy (Meta-BCE), which learns a separate feature representation for one-class classification, and One-Class Meta-Learning (OCML), which learns to generate one-class classifiers given standard multiclass feature representation. Both methods can augment any existing few-shot learning method without requiring retraining to work in a few-shot multiclass open-set setting without degrading its closed-set performance. We demonstrate the benefits and drawbacks of both methods in different problem settings and evaluate them on three standard benchmark datasets, miniImageNet, tieredImageNet, and Caltech-UCSD-Birds-200-2011, where they surpass the state-of-the-art methods in the few-shot multiclass open-set and few-shot one-class tasks.

Motivation & Objective

• To address the challenge of few-shot open-set classification, where standard methods fail due to limited per-class samples and closed-set assumptions.
• To develop meta-learning approaches that enable few-shot models to detect unknown classes without requiring separate background categories during training.
• To design methods that maintain high closed-set accuracy while adding robust open-set detection capabilities to existing few-shot learning frameworks.
• To enable effective few-shot one-class classification (n=1) where existing softmax-based methods fail.
• To ensure compatibility with any existing few-shot learning model by adding the methods as plug-in modules without retraining.

Proposed method

• Meta-BCE trains a separate feature representation for one-class classification using binary cross-entropy loss, optimized via meta-learning to generalize to unseen one-class examples.
• OCML learns to generate one-class classifiers from standard multiclass feature representations using a meta-learned classifier head, enabling open-set detection in few-shot settings.
• Both methods are trained as plug-in modules on top of existing few-shot learning models (e.g., FEAT), without modifying the base model's training process.
• The methods do not require background or unknown category data during training, unlike prior open-set methods that rely on such data.
• Meta-BCE is optimized for higher-shot settings (k ≥ 5), while OCML is tailored for low-shot (k=1) scenarios, based on empirical performance trade-offs.
• The approach leverages meta-learning to adapt to both few-shot one-class and multiclass open-set tasks, generalizing across varying n-way and k-shot configurations.

Experimental results

Research questions

• RQ1 Can meta-learning be effectively adapted to few-shot one-class classification, where only one class is present in the support set?
• RQ2 How can existing few-shot learning models be augmented to detect unknown classes without retraining or performance degradation?
• RQ3 Do methods that do not require background category data during training outperform those that do in few-shot open-set settings?
• RQ4 How does the performance of one-class detection vary with the number of support examples (k)?
• RQ5 Can the proposed methods generalize across different few-shot benchmarks and network architectures?

Key findings

• FEAT + OCML achieves 67.76% normalized accuracy on tieredImageNet in the 1-shot setting, outperforming all baselines.
• FEAT + Meta-BCE achieves 73.32% normalized accuracy on miniImageNet in the 5-shot setting, surpassing the state of the art.
• Meta-BCE achieves 70.98% normalized accuracy on miniImageNet with ResNet-18 and k=5, while OCML achieves 57.60% with k=1.
• The proposed methods do not degrade closed-set accuracy, unlike Entropic Open-Set Loss and Objectosphere, which show performance drops.
• OCML and Meta-BCE outperform softmax-based methods (e.g., PEELER, OpenMax) in the one-class setting, where such methods fail due to softmax limitations.
• On CUB-200-2011, FEAT + OCML achieves 60.21% normalized accuracy in 1-shot, and FEAT + Meta-BCE achieves 72.04% in 5-shot, both state-of-the-art results.