[Paper Review] Data-Driven Volumetric Image Generation from Surface Structures using a Patient-Specific Deep Leaning Model

Shaoyan Pan, Chih‐Wei Chang+6 more|PubMed|Apr 28, 2023

Medical Imaging Techniques and Applications21 references4 citations

TL;DR

This study proposes a patient-specific deep learning model that generates 3D volumetric CT images from zero-dose surface scans, using a generative adversarial network with hierarchical feature learning. The method achieves high fidelity synthetic CTs with a mean absolute error of 26.9 ± 4.1 Hounsfield units, PSNR of 39.1 ± 1.0 dB, and SSIM of 0.965 ± 0.011, enabling radiation-free, real-time image guidance for radiotherapy.

ABSTRACT

The advent of computed tomography significantly improves patients' health regarding diagnosis, prognosis, and treatment planning and verification. However, tomographic imaging escalates concomitant radiation doses to patients, inducing potential secondary cancer by 4%. We demonstrate the feasibility of a data-driven approach to synthesize volumetric images using patients' surface images, which can be obtained from a zero-dose surface imaging system. This study includes 500 computed tomography (CT) image sets from 50 patients. Compared to the ground truth CT, the synthetic images result in the evaluation metric values of 26.9 ± 4.1 Hounsfield units, 39.1 ± 1.0 dB, and 0.965 ± 0.011 regarding the mean absolute error, peak signal-to-noise ratio, and structural similarity index measure. This approach provides a data integration solution that can potentially enable real-time imaging, which is free of radiation-induced risk and could be applied to image-guided medical procedures.

Motivation & Objective

• To develop a data-driven method for generating 3D volumetric CT images from non-ionizing surface images to reduce patient radiation exposure.
• To address the challenge of generating detailed anatomical structures from sparse, low-dimensional surface data without prior anatomical priors.
• To enable real-time, radiation-free image guidance in radiotherapy, particularly for FLASH and image-guided procedures.
• To integrate patient-specific surface geometry with deep learning to infer hidden volumetric anatomy through learned feature correlations.
• To validate the model’s performance using clinical CT as ground truth and assess fidelity via standard image quality metrics.

Proposed method

• A generative adversarial network (GAN) with a hierarchical architecture is used to map 3D surface images to 3D volumetric CT images.
• A reconstruction network extracts feature maps from the surface image and transforms them into tensor representations for volumetric synthesis.
• A refinement network is trained to match the intensity distribution, noise level, and contrast resolution of clinically acquired CT scans.
• Surface curvature features are extracted and used as input modality, with magnitude of curvature shown to improve model generalization and reduce uncertainty.
• The model is trained end-to-end on 500 CT-surface image pairs from 50 patients, using patient-specific anatomical priors to enhance reconstruction fidelity.
• t-SNE and clustering analysis are used to evaluate the consistency and distribution of generated images across different patient groups.

Experimental results

Research questions

• RQ1 Can a deep learning model accurately reconstruct 3D volumetric CT images from 3D surface scans without using any prior anatomical information?
• RQ2 How does surface curvature influence the performance and uncertainty of the surface-to-volume image generation process?
• RQ3 To what extent can a data-driven, patient-specific model replicate the image quality of clinically acquired CT scans using only surface geometry?
• RQ4 Can this method enable real-time, radiation-free image guidance in radiotherapy, particularly for high-precision treatments like FLASH?
• RQ5 How does the integration of patient-specific surface data improve the generalization and robustness of volumetric image synthesis?

Key findings

• The proposed model achieves a mean absolute error (MAE) of 26.9 ± 4.1 Hounsfield units when comparing synthetic CTs to ground truth CTs.
• The peak signal-to-noise ratio (PSNR) of the synthetic images is 39.1 ± 1.0 dB, indicating high image quality and fidelity.
• The structural similarity index measure (SSIM) reaches 0.965 ± 0.011, demonstrating strong preservation of structural details.
• Patients with higher surface curvature variation show reduced model uncertainty, suggesting improved learning and generalization.
• The method enables radiation-free, real-time volumetric image generation, offering a potential solution for continuous image guidance in radiotherapy.
• The refinement network successfully preserves clinical CT noise levels and contrast resolution, enhancing realism and diagnostic utility.