[Paper Review] Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models

Qing Lyu, Ge Wang|arXiv (Cornell University)|Sep 24, 2022

Radiomics and Machine Learning in Medical Imaging53 citations

TL;DR

The paper adapts diffusion and score-matching generative models to synthesize CT images from MRI (T2-weighted) inputs, compares multiple sampling strategies against CNN/GAN baselines, and quantifies uncertainties via Monte Carlo sampling.

ABSTRACT

MRI and CT are most widely used medical imaging modalities. It is often necessary to acquire multi-modality images for diagnosis and treatment such as radiotherapy planning. However, multi-modality imaging is not only costly but also introduces misalignment between MRI and CT images. To address this challenge, computational conversion is a viable approach between MRI and CT images, especially from MRI to CT images. In this paper, we propose to use an emerging deep learning framework called diffusion and score-matching models in this context. Specifically, we adapt denoising diffusion probabilistic and score-matching models, use four different sampling strategies, and compare their performance metrics with that using a convolutional neural network and a generative adversarial network model. Our results show that the diffusion and score-matching models generate better synthetic CT images than the CNN and GAN models. Furthermore, we investigate the uncertainties associated with the diffusion and score-matching networks using the Monte-Carlo method, and improve the results by averaging their Monte-Carlo outputs. Our study suggests that diffusion and score-matching models are powerful to generate high quality images conditioned on an image obtained using a complementary imaging modality, analytically rigorous with clear explainability, and highly competitive with CNNs and GANs for image synthesis.

Motivation & Objective

• Motivate cross-modality CT-MRI synthesis to enable cost-effective multi-modality diagnostics and radiotherapy planning.
• Develop diffusion and score-matching based frameworks conditioned on MRI to generate synthetic CT images.
• Compare multiple sampling strategies (DDPM, SDE-based methods) against CNN and GAN baselines.
• Quantify predictive uncertainty in generated CT images using Monte Carlo averaging and variability maps.
• Provide analytical justification and empirical evidence for the effectiveness and explainability of diffusion-based approaches in cross-modality synthesis.

Proposed method

• Extend conditional DDPM to map MRI (T2w) to CT conditioned generation using a UNet denoiser.
• Adapt conditioning in diffusion and score-based models to MRI inputs and derive the conditional reverse processes.
• Implement and compare multiple sampling strategies: DDPM, Euler-Maruyama (EM), Prediction-Corrector (PC), and Ordinary Differential Equation (ODE) sampling.
• Train and evaluate using the Gold Atlas pelvis dataset with co-registered MRI-CT pairs.
• Quantify uncertainty by performing Monte Carlo sampling (10 runs) and averaging results to obtain DDPM-M, EM-M, PC-M, and ODE-M.

Experimental results

Research questions

• RQ1 Can diffusion and score-matching models conditioned on MRI produce CT-like images with high fidelity compared to CNN and GAN baselines?
• RQ2 How do different sampling strategies (DDPM, EM, PC, ODE) influence image quality and inference speed in MRI-to-CT synthesis?
• RQ3 What is the impact of MC-based averaging on image quality and model uncertainty for cross-modality synthesis?
• RQ4 How do diffusion-score models compare to traditional deep learning approaches in terms of realism, detail preservation, and artifact appearance in synthetic CTs?

Key findings

• Diffusion and score-matching models generate higher quality synthetic CT images than CNN and GAN baselines according to qualitative and quantitative metrics.
• Monte Carlo averaging (DDPM-M, EM-M, PC-M, ODE-M) yields superior SSIM and PSNR scores compared to single-sample outputs.
• Among sampling methods, ODE is fastest while PC is slowest, with DDPM and EM offering competitive quality.
• DDPM tends to show higher fidelity and uncertainty than some SDE-based methods, but averaging across methods reduces variability.
• Compared with CNNs (over-smoothed) and GANs (artifact-prone), diffusion-based approaches provide faithful details and fewer artifacts in synthetic CT images.