[Paper Review] Multi-Level Global Context Cross Consistency Model for Semi-Supervised Ultrasound Image Segmentation with Diffusion Model

Fenghe Tang, Jianrui Ding+3 more|arXiv (Cornell University)|May 16, 2023

Radiomics and Machine Learning in Medical Imaging14 citations

TL;DR

The paper introduces MGCC, a semi-supervised framework that uses a Latent Diffusion Model to generate unlabeled ultrasound images and a fully convolutional network with multi-level global context cross-consistency to improve segmentation. It shows improved performance on open breast ultrasound (BUSI) and private thyroid ultrasound (TUS) datasets.

ABSTRACT

Medical image segmentation is a critical step in computer-aided diagnosis, and convolutional neural networks are popular segmentation networks nowadays. However, the inherent local operation characteristics make it difficult to focus on the global contextual information of lesions with different positions, shapes, and sizes. Semi-supervised learning can be used to learn from both labeled and unlabeled samples, alleviating the burden of manual labeling. However, obtaining a large number of unlabeled images in medical scenarios remains challenging. To address these issues, we propose a Multi-level Global Context Cross-consistency (MGCC) framework that uses images generated by a Latent Diffusion Model (LDM) as unlabeled images for semi-supervised learning. The framework involves of two stages. In the first stage, a LDM is used to generate synthetic medical images, which reduces the workload of data annotation and addresses privacy concerns associated with collecting medical data. In the second stage, varying levels of global context noise perturbation are added to the input of the auxiliary decoder, and output consistency is maintained between decoders to improve the representation ability. Experiments conducted on open-source breast ultrasound and private thyroid ultrasound datasets demonstrate the effectiveness of our framework in bridging the probability distribution and the semantic representation of the medical image. Our approach enables the effective transfer of probability distribution knowledge to the segmentation network, resulting in improved segmentation accuracy. The code is available at https://github.com/FengheTan9/Multi-Level-Global-Context-Cross-Consistency.

Motivation & Objective

• Address the challenge of limited labeled ultrasound data by leveraging synthetic unlabeled data generated via a Latent Diffusion Model (LDM).
• Bridge diffusion probability distribution knowledge with semantic segmentation representations to improve model generalization.
• Introduce a fully convolutional network with multi-level global context cross-consistency to better capture global information in ultrasound images.
• Enhance feature transfer and robustness using multi-scale attention gates and ConvMixer-based global context modules.
• Evaluate the framework on open-source breast ultrasound (BUSI) and private thyroid ultrasound (TUS) datasets and compare with state-of-the-art SSL methods.

Proposed method

• Generate synthetic ultrasound images with a Latent Diffusion Model to create unlabeled data from labeled and unlabeled real data.
• Use a two-stage training setup where a shared encoder feeds into a main decoder and multiple auxiliary decoders with different global context perturbations.
• Integrate ConvMixer modules of varying lengths to mix distant spatial locations and obtain multi-level global context information.
• Incorporate multi-scale attention gates in skip-connections to select informative encoder features across receptive fields.
• Optimize a combined loss with supervised components for labeled data and an unsupervised consistency loss between main and auxiliary decoders on unlabeled data, with a Gaussian warming-up schedule.

Experimental results

Research questions

• RQ1 Can synthetic data from a diffusion model provide diffusion probability distribution knowledge useful for segmentation?
• RQ2 Does multi-level global context cross-consistency improve segmentation robustness to varying lesion positions and morphologies?
• RQ3 How does integrating ConvMixer-based global context and multi-scale gates affect semi-supervised ultrasound segmentation performance?
• RQ4 What is the impact of using unlabeled data from similar vs. target domains on segmentation performance?

Key findings

• MGCC achieves improved IoU, Recall, Precision, and F1 across self-domain experiments on BUSI and TUS compared to several SSL baselines.
• MGCC performs competitively with fully supervised methods on BUSI with fewer labeled samples.
• Using synthetic unlabeled data via LDM and SSL enhances diffusion probability knowledge transfer to the segmentation network.
• Auxiliary decoders with multi-level global context perturbations and consistency training improve segmentation robustness for lesions with diverse shapes.
• Experiments with similar-domain unlabeled data can degrade performance due to domain gaps, highlighting the importance of domain alignment.