[Paper Review] DiffMIC: Dual-Guidance Diffusion Network for Medical Image Classification

Yijun Yang, Huazhu Fu+3 more|arXiv (Cornell University)|Mar 19, 2023

Machine Learning in Healthcare4 citations

TL;DR

This paper proposes DiffMIC, the first diffusion-based model for general medical image classification, which uses dual-granularity conditional guidance and condition-specific MMD regularization to denoise images and learn robust semantic representations. It achieves state-of-the-art performance across three diverse medical imaging tasks—placental maturity grading, skin lesion classification, and diabetic retinopathy grading—outperforming existing methods by significant margins.

ABSTRACT

Diffusion Probabilistic Models have recently shown remarkable performance in generative image modeling, attracting significant attention in the computer vision community. However, while a substantial amount of diffusion-based research has focused on generative tasks, few studies have applied diffusion models to general medical image classification. In this paper, we propose the first diffusion-based model (named DiffMIC) to address general medical image classification by eliminating unexpected noise and perturbations in medical images and robustly capturing semantic representation. To achieve this goal, we devise a dual conditional guidance strategy that conditions each diffusion step with multiple granularities to improve step-wise regional attention. Furthermore, we propose learning the mutual information in each granularity by enforcing Maximum-Mean Discrepancy regularization during the diffusion forward process. We evaluate the effectiveness of our DiffMIC on three medical classification tasks with different image modalities, including placental maturity grading on ultrasound images, skin lesion classification using dermatoscopic images, and diabetic retinopathy grading using fundus images. Our experimental results demonstrate that DiffMIC outperforms state-of-the-art methods by a significant margin, indicating the universality and effectiveness of the proposed model. Our code will be publicly available at https://github.com/scott-yjyang/DiffMIC.

Motivation & Objective

• To address the challenge of noisy, blurry, and ambiguous medical images across diverse modalities in classification tasks.
• To explore the potential of diffusion probabilistic models beyond generative tasks and apply them to high-level vision, specifically medical image classification.
• To improve feature representation by denoising images through a stochastic diffusion process while preserving critical anatomical details.
• To enhance classification robustness by integrating global and local priors via dual-granularity conditional guidance.
• To learn disentangled, shared, and robust representations across image and patch-level features using condition-specific MMD regularization.

Proposed method

• A dual-granularity conditional guidance (DCG) strategy conditions each diffusion step on both global image-level and local region-of-interest (ROI)-level priors to improve regional attention and fine-grained feature learning.
• The model applies a denoising diffusion process on three inputs: the full image, global prior, and local prior, each with respective noise prediction heads in a U-Net architecture.
• Condition-specific Maximum-Mean Discrepancy (MMD) regularization is enforced during the forward diffusion process to maximize mutual information between image-level and patch-level latent representations.
• Noise prediction is trained via mean squared error (MSE) loss on the combined noisy embeddings, while MMD loss is applied to the predicted noise of global and local branches to align their distributions.
• The image encoder extracts feature embeddings, which are fused with the denoised latent features from the diffusion U-Net to produce final classification logits.
• The reverse diffusion process progressively denoises the features, enabling clearer, class-separated representations at inference time, visualized via t-SNE.

Experimental results

Research questions

• RQ1 Can a diffusion-based model effectively improve general medical image classification by denoising and enhancing semantic representations?
• RQ2 How does dual-granularity conditional guidance—using both global and local priors—improve attention and classification performance in medical images?
• RQ3 To what extent does condition-specific MMD regularization enhance feature disentanglement and robustness in the latent space during the diffusion process?
• RQ4 Does the proposed DiffMIC model generalize across diverse medical image modalities such as ultrasound, dermatoscopy, and fundus imaging?
• RQ5 How does the integration of diffusion modeling compare to conventional CNNs and vision transformers in terms of accuracy and robustness on imbalanced medical datasets?

Key findings

• On the PMG2000 dataset for placental maturity grading, DiffMIC achieves an accuracy of 0.931 and F1-score of 0.926, outperforming the second-best method (PVT) by 0.024 in accuracy and 0.024 in F1-score.
• On the HAM10000 dataset for skin lesion classification, DiffMIC improves accuracy by 0.019 and F1-score by 0.053 over the second-best method, ProCo.
• On the APTOS2019 dataset for diabetic retinopathy grading, DiffMIC achieves a 0.021 higher accuracy and 0.042 higher F1-score than ProCo, demonstrating strong performance on imbalanced data.
• Abllation studies confirm that each component—diffusion, DCG, and MMD regularization—contributes incrementally to performance, with the full model outperforming the baseline (ResNet18) by 0.052 in accuracy.
• t-SNE visualization shows that as the diffusion process progresses, feature embeddings become increasingly class-separated, indicating effective noise removal and representation refinement.
• The model demonstrates generalization across three distinct medical imaging modalities: ultrasound (placental), dermatoscopic (skin lesion), and fundus (diabetic retinopathy), confirming its universality.