[Paper Review] SdCT-GAN: Reconstructing CT from Biplanar X-Rays with Self-driven Generative Adversarial Networks

Shuangqin Cheng, Qingliang Chen+5 more|arXiv (Cornell University)|Sep 10, 2023

Generative Adversarial Networks and Image Synthesis4 citations

TL;DR

This paper proposes SdCT-GAN, a self-driven generative adversarial network that reconstructs high-quality 3D CT images from two orthogonal biplanar X-rays by integrating edge-aware feature learning via a Sobel Gradient Guider and a novel auto-encoder discriminator. The method achieves superior perceptual quality and texture detail preservation, outperforming state-of-the-art baselines in both qualitative and LPIPS-based quantitative evaluations.

ABSTRACT

Computed Tomography (CT) is a medical imaging modality that can generate more informative 3D images than 2D X-rays. However, this advantage comes at the expense of more radiation exposure, higher costs, and longer acquisition time. Hence, the reconstruction of 3D CT images using a limited number of 2D X-rays has gained significant importance as an economical alternative. Nevertheless, existing methods primarily prioritize minimizing pixel/voxel-level intensity discrepancies, often neglecting the preservation of textural details in the synthesized images. This oversight directly impacts the quality of the reconstructed images and thus affects the clinical diagnosis. To address the deficits, this paper presents a new self-driven generative adversarial network model (SdCT-GAN), which is motivated to pay more attention to image details by introducing a novel auto-encoder structure in the discriminator. In addition, a Sobel Gradient Guider (SGG) idea is applied throughout the model, where the edge information from the 2D X-ray image at the input can be integrated. Moreover, LPIPS (Learned Perceptual Image Patch Similarity) evaluation metric is adopted that can quantitatively evaluate the fine contours and textures of reconstructed images better than the existing ones. Finally, the qualitative and quantitative results of the empirical studies justify the power of the proposed model compared to mainstream state-of-the-art baselines.

Motivation & Objective

• To address the limitation of existing CT reconstruction methods that prioritize pixel-level accuracy over texture and structural detail preservation.
• To reduce radiation exposure and cost by enabling 3D CT reconstruction from only two 2D X-ray projections.
• To improve perceptual image quality by incorporating edge and texture information from input X-rays into the generative process.
• To develop a GAN-based framework that enhances fine-grained anatomical structures and contours in reconstructed CT volumes.
• To validate the superiority of the proposed method using both standard metrics (PSNR, SSIM) and perceptual metrics (LPIPS).

Proposed method

• The generator employs two parallel U-Net encoders with a shared SGG (Sobel Gradient Guider) module to extract and enhance edge features from the input biplanar X-rays during encoding.
• A fusion network combines features from the two U-Net branches before decoding to reconstruct the 3D CT volume.
• The discriminator $D_d$ is a novel auto-encoder-based structure that reconstructs the input CT volume and compares it with the ground truth to enforce perceptual consistency.
• The auxiliary discriminator $D_r$ is a simple decoder that computes reconstruction loss $L_{dReconstr}$ on ground truth CT, improving feature learning for both global and local structures.
• The model uses LPIPS (Learned Perceptual Image Patch Similarity) as a perceptual loss to better evaluate and optimize fine textures and contours.
• Adversarial training is conducted between the generator and both discriminators, enabling the generator to produce realistic, high-fidelity CT volumes.

Experimental results

Research questions

• RQ1 Can a self-driven GAN framework effectively reconstruct detailed 3D CT volumes from only two biplanar X-ray projections?
• RQ2 How does integrating Sobel edge information into the generator improve texture and contour fidelity in reconstructed CT images?
• RQ3 To what extent does using an auto-encoder-based discriminator enhance perceptual quality compared to standard patch-based discriminators?
• RQ4 Does LPIPS-based loss lead to better preservation of fine anatomical details than traditional metrics like PSNR and SSIM?
• RQ5 How does SdCT-GAN compare to state-of-the-art methods in terms of both quantitative metrics and visual quality of reconstructed CT volumes?

Key findings

• SdCT-GAN outperforms state-of-the-art methods such as X2CT-GAN and X2CT-CNN in preserving fine textures and geometric boundaries, as shown in qualitative comparisons.
• The proposed method achieves higher LPIPS scores than baselines, indicating better perceptual similarity to ground truth CT volumes.
• The Sobel Gradient Guider significantly improves edge preservation, particularly in regions with complex anatomical structures like the spine.
• The auto-encoder discriminator structure enables more effective feature learning, leading to sharper and more realistic CT reconstructions.
• In PSNR and SSIM comparisons, SdCT-GAN achieves competitive performance, but its true advantage lies in perceptual quality, especially in texture and contour accuracy.
• Empirical results demonstrate that SdCT-GAN effectively reduces blurring and artifacts commonly seen in methods relying solely on MSE or L2 loss.