[Paper Review] Male pelvic synthetic CT generation from T1-weighted MRI using 2D and 3D convolutional neural networks

Jie Fu, Yingli Yang+4 more|arXiv (Cornell University)|Feb 28, 2018

Advanced X-ray and CT ImagingEngineering50 references44 citations

TL;DR

The paper develops and compares 2D and 3D CNN models to generate pelvic synthetic CTs from T1-weighted MR images, evaluating accuracy with voxel-wise and bone-specific metrics across 20 male patients.

ABSTRACT

To achieve magnetic resonance (MR)-only radiotherapy, a method needs to be employed to estimate a synthetic CT (sCT) for generating electron density maps and patient positioning reference images. We investigated 2D and 3D convolutional neural network (CNN) methods to generate a male pelvic sCT using a T1-weighted MR image. A retrospective study was performed using CTs and T1-weighted MR images of 20 prostate cancer patients. The proposed 2D CNN model, which contained 27 convolutional layers, was modified from the SegNet for better performance. 3D version of the CNN model was also developed. Both CNN models were trained from scratch to map intensities of T1-weighted MR images to CT Hounsfield Unit (HU) values. Each sCT was generated in a five-fold-cross-validation framework and compared with the corresponding CT using voxel-wise mean absolute error (MAE), and dice similarity coefficient (DSC), recall, and precision for bony structures. Wilcoxon signed-rank tests were performed to evaluate the differences between the both models. The MAE averaged across all patients were 40.5 $\pm$ 5.4 HU and 37.6 $\pm$ 5.1 HU for the 2D and 3D CNN models, respectively. The DSC, recall, and precision of the bony structures were 0.81 $\pm$ 0.04, 0.85 $\pm$ 0.04, and 0.77 $\pm$ 0.09 for the 2D CNN model, and 0.82 $\pm$ 0.04, 0.84 $\pm$ 0.04, and 0.80 $\pm$ 0.08 for the 3D CNN model, respectively. P values of the Wilcoxon signed-rank tests were less than 0.05 except for recall, which was 0.6. The 2D and 3D CNN models generated accurate pelvic sCTs for the 20 patients using T1-weighted MR images. The evaluation metrics and statistical tests indicated that the 3D model was able to generate sCTs with better MAE, bone DSC, and bone precision. The accuracy of the dose calculation and patient positioning using generated sCTs will be tested and compared for the two models in the future.

Motivation & Objective

• Demonstrate feasibility of MR-only radiotherapy by generating sCTs from MR images without manual bone segmentation or extra sequences.
• Compare 2D and 3D CNN architectures for pelvic sCT generation from T1-weighted MRI.
• Evaluate sCT accuracy using voxel-wise MAE across body, soft tissue, and bone regions, plus bone-specific DSC, recall, and precision.
• Assess training efficiency and practical considerations for implementing MR-only workflow in pelvis.

Proposed method

• Train from-scratch 2D and 3D CNNs modified from SegNet to map T1-weighted MRI intensities to CT Hounsfield Units.
• Use instance normalization, residual shortcuts, and fractionally-strided convolutions in the 2D model; extend the same architecture to 3D.
• Preprocess with N4 bias correction and histogram standardization; deformably register CT to MRI to create deformed CT for supervision.
• Train with five-fold cross-validation; optimize using Adam with a mean absolute error loss computed inside a body mask.
• Apply on-the-fly data augmentation (shifts/rotations) to improve generalization.
• Evaluate via MAE in whole body, soft tissue, and bone regions, plus DSC/recall/precision for bone.
• Compare 2D and 3D models using Wilcoxon signed-rank tests.

Experimental results

Research questions

• RQ1 Can a pelvic synthetic CT be accurately generated from a standard T1-weighted MR image without CT-or manual bone segmentation?
• RQ2 Do 2D and 3D CNN architectures differ in accuracy for pelvic sCT generation, particularly for bone and soft-tissue regions?
• RQ3 Is MR-only radiotherapy planning feasible with acceptable MAE and bone-delineation metrics using these CNN approaches?

Key findings

• Both 2D and 3D models produced accurate pelvic sCTs from T1-weighted MR images across 20 patients.
• 3D model achieved better MAE (37.6 ± 5.1 HU vs. 40.5 ± 5.4 HU for 2D) and slightly higher bone DSC/precision.
• Maximum whole-body MAEs were 56.5 HU (2D) and 53.1 HU (3D).
• Bone-region DSCs were 0.81 ± 0.04 (2D) and 0.82 ± 0.04 (3D); recall 0.85 ± 0.04 (2D) vs 0.84 ± 0.04 (3D); precision 0.77 ± 0.09 (2D) vs 0.80 ± 0.08 (3D).
• Wilcoxon tests showed significant differences for MAE and bone metrics (p < 0.05) except recall (p = 0.60).
• Generation time per sCT volume was ~5.5 seconds for both models.