[Paper Review] Latent Diffusion Model for Medical Image Standardization and Enhancement

Md. Selim, Jie Zhang+5 more|arXiv (Cornell University)|Oct 8, 2023

Radiomics and Machine Learning in Medical Imaging4 citations

TL;DR

This paper proposes DiffusionCT, a latent diffusion model that standardizes and enhances medical CT images by learning to transform non-standard, scanner-variant distributions into a consistent, standardized form. By training a U-Net-based encoder-decoder in latent space with a DDPM at the bottleneck, it achieves superior image standardization and noise reduction, outperforming GAN-based methods in preserving structural integrity and improving radiomic feature consistency.

ABSTRACT

Computed tomography (CT) serves as an effective tool for lung cancer screening, diagnosis, treatment, and prognosis, providing a rich source of features to quantify temporal and spatial tumor changes. Nonetheless, the diversity of CT scanners and customized acquisition protocols can introduce significant inconsistencies in texture features, even when assessing the same patient. This variability poses a fundamental challenge for subsequent research that relies on consistent image features. Existing CT image standardization models predominantly utilize GAN-based supervised or semi-supervised learning, but their performance remains limited. We present DiffusionCT, an innovative score-based DDPM model that operates in the latent space to transform disparate non-standard distributions into a standardized form. The architecture comprises a U-Net-based encoder-decoder, augmented by a DDPM model integrated at the bottleneck position. First, the encoder-decoder is trained independently, without embedding DDPM, to capture the latent representation of the input data. Second, the latent DDPM model is trained while keeping the encoder-decoder parameters fixed. Finally, the decoder uses the transformed latent representation to generate a standardized CT image, providing a more consistent basis for downstream analysis. Empirical tests on patient CT images indicate notable improvements in image standardization using DiffusionCT. Additionally, the model significantly reduces image noise in SPAD images, further validating the effectiveness of DiffusionCT for advanced imaging tasks.

Motivation & Objective

• To address the challenge of inconsistent radiomic features across CT scans from different scanners and protocols.
• To develop a post-processing framework that standardizes existing CT images without requiring protocol changes.
• To improve image quality by reducing noise in low-signal imaging modalities like TR-LSCI.
• To evaluate the performance of diffusion models, particularly DDPMs, in latent space for medical image standardization.
• To provide a robust, generalizable solution for harmonizing medical imaging data across multi-site and multi-scanner studies.

Proposed method

• A U-Net-based encoder-decoder network is pre-trained independently to learn the latent representation of input CT images without the DDPM component.
• A score-based DDPM is then trained in the latent space at the bottleneck, with the encoder-decoder weights frozen.
• The final model generates standardized images by passing the latent representation through the decoder after DDPM-based denoising.
• The model is trained using a denoising score matching objective in the latent space, minimizing the difference between predicted and true noise.
• The approach is evaluated on both clinical CT scans and TR-LSCI phantom images, using metrics like SSIM, PSNR, and CCC.
• A ResNet-18 encoder is used for feature extraction, with future work considering VGG and vanilla U-Net architectures.

Experimental results

Research questions

• RQ1 Can a latent diffusion model effectively standardize CT images across diverse scanners and acquisition protocols?
• RQ2 How does the performance of a DDPM-based approach compare to GAN-based methods in medical image standardization?
• RQ3 Does training the DDPM in the latent space preserve structural details while improving feature consistency?
• RQ4 Can the model effectively reduce noise in low-signal imaging modalities such as TR-LSCI?
• RQ5 What is the impact of the model on radiomic feature concordance, particularly for texture features like GLCM?

Key findings

• DiffusionCT achieved a concordance correlation coefficient (CCC) of 0.86 for GLCM features, significantly outperforming GAN-based models and vanilla DDPM.
• The model improved SSIM from 0.44 to 0.77 and PSNR from 12.50 to 23.75 when denoising TR-LSCI images, with results close to ground truth.
• The latent-space DDPM approach matched the performance of image-space DDPM while being more compact and efficient.
• The model successfully reduced inter-scanner variability in texture features, enhancing reliability for downstream radiomics analysis.
• Empirical results confirmed that the model preserves anatomical details while generating standardized images with consistent radiomic features.
• The approach demonstrates strong potential for multi-site medical imaging studies by enabling feature harmonization without protocol standardization.