[Paper Review] OpenCoS: Contrastive Semi-supervised Learning for Handling Open-set Unlabeled Data

Jongjin Park, Sukmin Yun+2 more|arXiv (Cornell University)|Jun 29, 2021

Domain Adaptation and Few-Shot Learning36 references7 citations

TL;DR

OpenCoS proposes a contrastive semi-supervised learning framework that effectively handles open-set unlabeled data—where unlabeled data contains out-of-class samples not present in the labeled set—by leveraging self-supervised contrastive representations to detect in- and out-of-class samples. It assigns one-hot pseudo-labels to confident in-class samples and soft-labels to out-of-class samples via auxiliary batch normalization, significantly improving performance over state-of-the-art SSL methods under class-distribution mismatch, achieving 69.38% accuracy on CIFAR-10 with only 40 labels when Tiny-ImageNet is used as out-of-class data.

ABSTRACT

Semi-supervised learning (SSL) has been a powerful strategy to incorporate few labels in learning better representations. In this paper, we focus on a practical scenario that one aims to apply SSL when unlabeled data may contain out-of-class samples - those that cannot have one-hot encoded labels from a closed-set of classes in label data, i.e., the unlabeled data is an open-set. Specifically, we introduce OpenCoS, a simple framework for handling this realistic semi-supervised learning scenario based upon a recent framework of self-supervised visual representation learning. We first observe that the out-of-class samples in the open-set unlabeled dataset can be identified effectively via self-supervised contrastive learning. Then, OpenCoS utilizes this information to overcome the failure modes in the existing state-of-the-art semi-supervised methods, by utilizing one-hot pseudo-labels and soft-labels for the identified in- and out-of-class unlabeled data, respectively. Our extensive experimental results show the effectiveness of OpenCoS under the presence of out-of-class samples, fixing up the state-of-the-art semi-supervised methods to be suitable for diverse scenarios involving open-set unlabeled data.

Motivation & Objective

• To address the practical limitation of semi-supervised learning (SSL) when unlabeled data contains out-of-class samples not present in the labeled set.
• To overcome the failure modes of existing SSL methods, which suffer from entropy minimization and consistency regularization when mislabeling out-of-class samples.
• To propose a generalizable framework that leverages unsupervised representations to detect and appropriately label both in-class and out-of-class unlabeled samples.
• To improve model generalization and performance in realistic SSL scenarios with class-distribution mismatch, especially under low-label regimes.

Proposed method

• Uses SimCLR-based self-supervised contrastive learning to pre-train representations on both labeled and unlabeled data, enabling effective detection of out-of-class samples.
• Applies a detection criterion based on contrastive representation similarity to distinguish in-class from out-of-class unlabeled samples.
• Assigns one-hot pseudo-labels to the top-k most confident in-class unlabeled samples for integration into the labeled dataset.
• Utilizes soft-labels derived from the model’s posterior predictive distribution for out-of-class samples, improving robustness to distribution shift.
• Introduces auxiliary batch normalization layers to decouple normalization statistics, mitigating the negative impact of out-of-class samples on feature statistics.
• Integrates the framework with existing SSL methods like ReMixMatch and FixMatch, enabling plug-and-play enhancement under open-set conditions.

Experimental results

Research questions

• RQ1 Can self-supervised contrastive representation learning effectively detect out-of-class samples in unlabeled data under class-distribution mismatch?
• RQ2 How can out-of-class samples be meaningfully utilized in semi-supervised learning instead of being discarded or mislabeled?
• RQ3 Does soft-labeling of out-of-class samples, informed by contrastive representations, improve generalization and accuracy compared to standard pseudo-labeling?
• RQ4 To what extent does OpenCoS improve state-of-the-art SSL methods under realistic open-set unlabeled data settings?
• RQ5 Can the proposed framework be generalized across different backbone architectures and pre-training schemes, such as ViT with DINO?

Key findings

• OpenCoS improves the test accuracy of ReMixMatch-ft from 22.53% to 58.02% on CIFAR-10 with 40 labeled samples and Tiny-ImageNet as out-of-class data.
• On the same benchmark, OpenCoS achieves 69.38% accuracy with only 40 labels (4 per class), outperforming the SOTA baseline OpenMatch, which achieves 62.71%.
• The ablation study confirms that each component—detection, auxiliary loss, auxiliary BNs, and top-k pseudo-labeling—contributes incrementally to performance, with the full OpenCoS model reaching 58.02%.
• When combined with DINO-pretrained ViT, OpenCoS improves FixMatch-ft accuracy on the Food super-class from 66.95% to 72.00%, demonstrating compatibility with vision transformers.
• The soft-labeling scheme successfully captures semantic similarity between out-of-class and in-class samples (e.g., animals resembling dogs), assigning higher confidence to such cases, while remaining close to uniform for unrelated samples.
• The use of auxiliary batch normalization layers significantly improves robustness, reducing the negative impact of out-of-class samples on batch statistics.