[Paper Review] Difficulty-aware Meta-learning for Rare Disease Diagnosis

Xiaomeng Li, Lequan Yu+4 more|arXiv (Cornell University)|Jun 30, 2019

AI in cancer detection11 references4 citations

TL;DR

This paper proposes a difficulty-aware meta-learning (DAML) method to improve few-shot classification of rare diseases using only five labeled samples per class. By dynamically down-weighting well-learned (easy) tasks and focusing optimization on hard tasks during meta-training, DAML achieves an AUC of 83.3% on unseen rare skin disease classes in the ISIC 2018 dataset and 82.67% on real-world Dermofit Image Library data, demonstrating strong transferability and clinical potential.

ABSTRACT

Rare diseases have extremely low-data regimes, unlike common diseases with large amount of available labeled data. Hence, to train a neural network to classify rare diseases with a few per-class data samples is very challenging, and so far, catches very little attention. In this paper, we present a difficulty-aware meta-learning method to address rare disease classifications and demonstrate its capability to classify dermoscopy images. Our key approach is to first train and construct a meta-learning model from data of common diseases, then adapt the model to perform rare disease classification.To achieve this, we develop the difficulty-aware meta-learning method that dynamically monitors the importance of learning tasks during the meta-optimization stage. To evaluate our method, we use the recent ISIC 2018 skin lesion classification dataset, and show that with only five samples per class, our model can quickly adapt to classify unseen classes by a high AUC of 83.3%. Also, we evaluated several rare disease classification results in the public Dermofit Image Library to demonstrate the potential of our method for real clinical practice.

Motivation & Objective

• To address the challenge of rare disease diagnosis under extremely low-data regimes, where only a few labeled samples per class are available.
• To improve few-shot generalization in medical image classification by enhancing meta-learning optimization through dynamic task weighting.
• To reduce overfitting in fine-tuning by focusing meta-optimization on hard, underrepresented tasks during training.
• To validate the method on real-world clinical datasets, such as the Dermofit Image Library, to demonstrate practical applicability.
• To explore the feasibility of applying the method in privacy-sensitive settings like federated learning.

Proposed method

• The method employs a meta-learning framework that trains a shared initialization on common disease tasks, enabling fast adaptation to rare disease tasks via few gradient steps.
• A difficulty-aware loss function is introduced, which dynamically modulates the contribution of each task based on its current learning difficulty, down-weighting easy tasks and emphasizing hard ones.
• The dynamic modulating function uses a learnable parameter η to control the degree of down-weighting, with optimal performance observed at η=5.
• The model architecture uses 4 convolutional blocks with 64 feature maps and heavy data augmentation (rotation, flipping, scaling, cropping) to improve robustness.
• The method is evaluated using a meta-training phase on common diseases and a meta-testing phase on unseen rare disease classes with only five samples per class.
• The approach is compared against MAML, task sampling, and metric-based baselines, showing consistent superiority across all settings.

Experimental results

Research questions

• RQ1 Can meta-learning be enhanced to better handle rare disease classification under extremely low-data regimes with only five samples per class?
• RQ2 Does dynamically down-weighting easy tasks during meta-optimization improve generalization to unseen rare disease classes?
• RQ3 How does the difficulty-aware loss function compare to standard meta-learning and task sampling methods in few-shot medical image classification?
• RQ4 Can the proposed method achieve high performance on real-world clinical datasets like the Dermofit Image Library?
• RQ5 Is the method suitable for deployment in privacy-preserving learning frameworks such as federated learning?

Key findings

• On the ISIC 2018 dataset, the proposed DAML method achieved an AUC of 83.3% when classifying three unseen rare skin disease classes with only five training samples per class.
• On the public Dermofit Image Library, the method achieved an average AUC of 82.67% under the five-samples-per-class setting, outperforming MAML and task sampling baselines.
• Ablation studies showed that the optimal difficulty-aware loss parameter η=5, yielding the highest performance, while higher values (e.g., η=7) led to slight degradation.
• Using residual blocks led to overfitting, whereas simpler convolutional blocks with 64 filters and data augmentation yielded better generalization.
• Heavy data augmentation improved AUC from 79.3% to 81.2%, but the most significant gains came from the difficulty-aware optimization, not augmentation.
• The method outperformed MAML and task sampling methods across all sample settings (1, 3, 5 samples), confirming its robustness in low-data regimes.