[Paper Review] Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label Learning

Mingkun Xie, Jia‐Hao Xiao+4 more|arXiv (Cornell University)|May 4, 2023

Text and Document Classification Technologies4 citations

TL;DR

This paper proposes Class-Aware Pseudo-Labeling (CAP), a novel method for semi-supervised multi-label learning that assigns pseudo-labels in a class-aware manner using class-distribution-aware thresholds. By estimating the true class distribution from labeled data and enforcing alignment through regularized learning, CAP significantly improves pseudo-labeling accuracy and achieves state-of-the-art performance on benchmark datasets, especially with limited labeled data.

ABSTRACT

Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data. However, in the context of semi-supervised multi-label learning (SSMLL), conventional pseudo-labeling methods encounter difficulties when dealing with instances associated with multiple labels and an unknown label count. These limitations often result in the introduction of false positive labels or the neglect of true positive ones. To overcome these challenges, this paper proposes a novel solution called Class-Aware Pseudo-Labeling (CAP) that performs pseudo-labeling in a class-aware manner. The proposed approach introduces a regularized learning framework incorporating class-aware thresholds, which effectively control the assignment of positive and negative pseudo-labels for each class. Notably, even with a small proportion of labeled examples, our observations demonstrate that the estimated class distribution serves as a reliable approximation. Motivated by this finding, we develop a class-distribution-aware thresholding strategy to ensure the alignment of pseudo-label distribution with the true distribution. The correctness of the estimated class distribution is theoretically verified, and a generalization error bound is provided for our proposed method. Extensive experiments on multiple benchmark datasets confirm the efficacy of CAP in addressing the challenges of SSMLL problems.

Motivation & Objective

• Address the challenge of unreliable pseudo-labeling in semi-supervised multi-label learning (SSMLL), where conventional instance-aware methods struggle with unknown label counts and class imbalance.
• Overcome the limitations of existing methods that fail under SSMLL settings due to assumptions about missing labels or fixed thresholds.
• Develop a method that leverages unlabeled data effectively by estimating the true class distribution and aligning pseudo-label distributions with it.
• Ensure robust performance even with a small number of labeled examples by using a regularized learning framework with class-aware thresholds.

Proposed method

• Propose a class-aware pseudo-labeling strategy that assigns positive and negative pseudo-labels per class using thresholds informed by the estimated class distribution.
• Introduce a regularized learning framework that controls the number of positive and negative pseudo-labels per class based on class-aware thresholds.
• Estimate the true class distribution from labeled data, which is theoretically justified as a reliable approximation under the proposed observation.
• Implement a class-distribution-aware thresholding strategy to align the distribution of pseudo-labels with the true underlying class distribution.
• Theoretical analysis proves the correctness of the estimated class distribution and provides a generalization error bound for CAP.
• Use a two-stage training process: first estimate class distribution from labeled data, then apply thresholding to generate pseudo-labels for unlabeled data.

Experimental results

Research questions

• RQ1 Can a class-aware pseudo-labeling strategy improve performance in semi-supervised multi-label learning compared to instance-aware methods?
• RQ2 How accurately can the true class distribution be estimated from a small set of labeled data in multi-label settings?
• RQ3 Does aligning pseudo-label distribution with the true class distribution lead to better generalization and reduced false positives/negatives?
• RQ4 How does the use of reliable intervals (controlled by η₁ and η₀) affect model performance in the presence of label imbalance?

Key findings

• CAP achieves state-of-the-art performance across all benchmark datasets (VOC, COCO, NUS), significantly outperforming existing methods, especially when labeled data is scarce.
• On VOC and COCO, CAP achieves the highest mAP scores, with improvements of up to 5.2% over the next best baseline under low-label regimes (p=0.05).
• The method outperforms both instance-aware pseudo-labeling baselines and two strong SSL methods, even after extensive hyperparameter tuning and architectural modifications.
• At the first training epoch, CAP already outperforms other methods in CF1 score despite identical model predictions, demonstrating its superior ability to estimate class proportions.
• Discarding unreliable positive pseudo-labels (via η₁ < 1) improves performance, while removing negative pseudo-labels degrades performance due to the inherent positive-negative imbalance in multi-label data.
• Theoretical analysis confirms the correctness of the estimated class distribution and provides a generalization error bound that depends on pseudo-labeling quality.