[Paper Review] ACPL: Anti-curriculum Pseudo-labelling for Semi-supervised Medical Image Classification

Fengbei Liu, Yu Tian+4 more|arXiv (Cornell University)|Nov 25, 2021

Infrastructure Maintenance and Monitoring4 citations

TL;DR

ACPL proposes a novel semi-supervised learning method for medical image classification that selects highly informative unlabelled samples based on cross-distribution informativeness to mitigate class imbalance, and uses an ensemble of model and KNN predictions via informative mixup to improve pseudo-label accuracy. It achieves state-of-the-art performance on both multi-label (Chest X-ray14) and multi-class (ISIC2018) benchmarks by addressing confirmation bias and training imbalance without requiring class-wise thresholds.

ABSTRACT

Effective semi-supervised learning (SSL) in medical image analysis (MIA) must address two challenges: 1) work effectively on both multi-class (e.g., lesion classification) and multi-label (e.g., multiple-disease diagnosis) problems, and 2) handle imbalanced learning (because of the high variance in disease prevalence). One strategy to explore in SSL MIA is based on the pseudo labelling strategy, but it has a few shortcomings. Pseudo-labelling has in general lower accuracy than consistency learning, it is not specifically designed for both multi-class and multi-label problems, and it can be challenged by imbalanced learning. In this paper, unlike traditional methods that select confident pseudo label by threshold, we propose a new SSL algorithm, called anti-curriculum pseudo-labelling (ACPL), which introduces novel techniques to select informative unlabelled samples, improving training balance and allowing the model to work for both multi-label and multi-class problems, and to estimate pseudo labels by an accurate ensemble of classifiers (improving pseudo label accuracy). We run extensive experiments to evaluate ACPL on two public medical image classification benchmarks: Chest X-Ray14 for thorax disease multi-label classification and ISIC2018 for skin lesion multi-class classification. Our method outperforms previous SOTA SSL methods on both datasets

Motivation & Objective

• Address the challenge of class imbalance in semi-supervised medical image classification, especially for rare disease classes.
• Overcome the limitations of traditional pseudo-labelling, such as confirmation bias and reliance on fixed confidence thresholds.
• Enable effective semi-supervised learning for both multi-class and multi-label medical image problems.
• Improve pseudo-label accuracy by combining deep model predictions with KNN classification using sample informativeness as a weighting criterion.
• Develop a stable and balanced training process by selecting unlabelled samples that are far from the labelled distribution, promoting minority class learning.

Proposed method

• Introduce a cross-distribution sample informativeness measure to identify unlabelled samples that are far from the labelled anchor set, prioritizing those likely to belong to minority classes.
• Propose informative mixup, a pseudo-labelling mechanism that combines model and KNN predictions using a density-based weight derived from the anchor set to improve label accuracy and reduce confirmation bias.
• Design an anchor set purification strategy that selects only the most informative pseudo-labelled samples to update the anchor set, enhancing KNN classifier reliability over training epochs.
• Use a dynamic anchor set that evolves by incorporating high-informativeness pseudo-labellings, improving the stability and accuracy of subsequent KNN predictions.
• Apply a curriculum-like learning strategy that avoids selecting easy, majority-class samples by focusing on informative, distributionally distant unlabelled samples.
• Implement a KNN classifier guided by sample informativeness to provide robust pseudo-labels, especially in multi-label settings with complex class correlations.

(a) Diagram of our ACPL (top) and traditional pseudo-label SSL (bottom)

Experimental results

Research questions

• RQ1 Can selecting unlabelled samples based on their distributional distance from labelled data improve model generalization in imbalanced medical image classification?
• RQ2 How does combining model predictions with KNN classification using informativeness-weighted mixup affect pseudo-label accuracy and model robustness?
• RQ3 To what extent does the proposed method mitigate confirmation bias in semi-supervised medical image learning?
• RQ4 Can the method generalize across both multi-class and multi-label medical image benchmarks without requiring problem-specific threshold tuning?
• RQ5 Does the anchor set purification mechanism enhance the stability and performance of the KNN-based pseudo-labelling component?

Key findings

• ACPL achieves a test AUC of 74.44 on the multi-label Chest X-ray14 dataset with only 2% labelled data, outperforming previous SOTA methods.
• On the multi-class ISIC2018 benchmark, ACPL achieves state-of-the-art performance, demonstrating effectiveness across different problem types.
• The informative mixup strategy improves pseudo-label accuracy by at least 1.5% over baseline methods, including model-only, KNN-only, and random alpha mixup.
• Sample selection based on informativeness reduces the disparity between minority and majority class representation, increasing minority class sample percentages from 5–10% to nearly 30%.
• The anchor set purification mechanism stabilizes performance across different KNN neighbor counts, reducing AUC variance from 2% to 1%.
• Ablation studies confirm that both informative sample selection and the informative mixup mechanism are critical for performance gains, with each component contributing significantly to final accuracy.

(b) Imbalanced distribution on multi-label Chest X-ray14 [ 39 ] (left) and multi-class ISIC2018 [ 36 ] (right)