[Paper Review] Deep Learning from Small Amount of Medical Data with Noisy Labels: A Meta-Learning Approach

Görkem Algan, İlkay Ulusoy+3 more|arXiv (Cornell University)|Oct 14, 2020

Machine Learning and Data Classification28 references4 citations

TL;DR

This paper proposes a meta-learning framework to improve deep learning performance on small, noisy medical datasets by leveraging a small amount of clean 'meta-data' to guide learning from noisy labels. The method uses soft-label prediction during meta-training to reduce overfitting to label noise, achieving 91.4% test accuracy on a retinopathy of prematurity dataset with 68% label noise—outperforming standard cross-entropy training and existing methods on smaller datasets.

ABSTRACT

Computer vision systems recently made a big leap thanks to deep neural networks. However, these systems require correctly labeled large datasets in order to be trained properly, which is very difficult to obtain for medical applications. Two main reasons for label noise in medical applications are the high complexity of the data and conflicting opinions of experts. Moreover, medical imaging datasets are commonly tiny, which makes each data very important in learning. As a result, if not handled properly, label noise significantly degrades the performance. Therefore, a label-noise-robust learning algorithm that makes use of the meta-learning paradigm is proposed in this article. The proposed solution is tested on retinopathy of prematurity (ROP) dataset with a very high label noise of 68%. Results show that the proposed algorithm significantly improves the classification algorithm's performance in the presence of noisy labels.

Motivation & Objective

• Address the challenge of training deep learning models on small medical datasets with high label noise due to expert disagreement and data complexity.
• Overcome the limitations of conventional loss functions and data-cleaning methods that fail on small, noisy datasets.
• Develop a meta-learning framework that uses minimal verified clean data to guide the model in learning from noisy labels effectively.
• Improve generalization and prevent overfitting to noisy labels in medical image classification, especially when data is scarce.
• Demonstrate state-of-the-art performance on a small, highly noisy retinopathy of prematurity dataset.

Proposed method

• The framework employs a two-loop meta-learning architecture: a conventional training loop and a meta-training loop to optimize hyperparameters.
• During meta-training, the model learns to predict soft-label representations of the noisy training data using a small set of verified clean examples (meta-data).
• The base classifier is trained on predicted soft-labels instead of ground-truth noisy labels, reducing memorization of incorrect labels.
• The meta-objective is optimized using a weighted combination of meta-loss and base-loss, with hyperparameters K=10, α=0.5, β=4000.
• A pre-training step on a larger diabetic retinopathy dataset improves feature learning before fine-tuning on the ROP dataset.
• Stochastic gradient descent with momentum and learning rate scheduling is used, with 10 warm-up epochs followed by 20 meta-training epochs.

Experimental results

Research questions

• RQ1 Can meta-learning effectively improve model generalization when training on small medical datasets with extreme label noise?
• RQ2 How does using a small amount of clean meta-data enhance learning from noisy labels in medical image classification?
• RQ3 To what extent can a soft-label prediction mechanism reduce overfitting to noisy labels compared to standard cross-entropy training?
• RQ4 Does the proposed method outperform conventional and existing state-of-the-art approaches on small, noisy medical datasets?
• RQ5 Can the model maintain high test accuracy while avoiding 100% training accuracy, indicating robustness to label noise?

Key findings

• The proposed method achieved 91.4% test accuracy on the ROP dataset with 68% label noise, outperforming standard cross-entropy training (86.3% and 90.3%).
• Training accuracy was limited to 86.23% in the proposed method, indicating effective prevention of overfitting to noisy labels, unlike cross-entropy which reached 100%.
• The method achieved comparable performance to prior works that used significantly larger training sets (5.5k–8.5k images) while training on only 1.9k images.
• Pre-training on a larger diabetic retinopathy dataset improved performance by 4% over direct training on the ROP dataset alone.
• The framework successfully leveraged minimal clean meta-data to extract useful knowledge from noisy data, demonstrating effectiveness in low-data, high-noise regimes.
• The model maintained robustness even with extreme label noise, suggesting practical utility for real-world medical imaging applications.