[Paper Review] Whole-Body Lesion Segmentation in 18F-FDG PET/CT

Jia Zhang, Yukun Huang+2 more|arXiv (Cornell University)|Sep 16, 2022

Medical Imaging Techniques and Applications4 citations

TL;DR

This paper proposes a joint 2D-3D nnUNet model with a ResNet18 backbone for whole-body lesion segmentation in 18F-FDG PET/CT scans, combining axial-plane sensitivity from 2D networks with 3D volumetric context from 3D ResNet-based networks. The method achieved state-of-the-art performance on the AutoPet Challenge, with a Dice score of 0.79 on the total cohort and reduced false positives and false negatives through weighted fusion and post-processing.

ABSTRACT

There has been growing research interest in using deep learning based method to achieve fully automated segmentation of lesion in Positron emission tomography computed tomography(PET CT) scans for the prognosis of various cancers. Recent advances in the medical image segmentation shows the nnUNET is feasible for diverse tasks. However, lesion segmentation in the PET images is not straightforward, because lesion and physiological uptake has similar distribution patterns. The Distinction of them requires extra structural information in the CT images. The present paper introduces a nnUNet based method for the lesion segmentation task. The proposed model is designed on the basis of the joint 2D and 3D nnUNET architecture to predict lesions across the whole body. It allows for automated segmentation of potential lesions. We evaluate the proposed method in the context of AutoPet Challenge, which measures the lesion segmentation performance in the metrics of dice score, false-positive volume and false-negative volume.

Motivation & Objective

• To develop a fully automated, deep learning-based method for whole-body lesion segmentation in 18F-FDG PET/CT scans to support cancer prognosis and treatment planning.
• To address the challenge of distinguishing malignant lesions from physiological FDG uptake in organs like the brain, bladder, and kidneys using structural information from CT.
• To improve segmentation accuracy by combining 2D and 3D network architectures within the nnUNet framework to leverage both axial-plane sensitivity and 3D volumetric context.
• To reduce false positives and false negatives through a weighted fusion strategy and a novel post-processing method.

Proposed method

• The proposed method uses a joint 2D-3D nnUNet architecture, where 2D UNet and 3D full-resolution ResUNet (3D F-Res) are trained independently and their predictions are fused using a weighted average.
• The 3D F-Res model integrates ResNet18 as the encoder backbone, enabling deeper network connections and improved information flow through residual units.
• The 2D UNet processes axial slices with a patch size of 448×448, capturing fine-grained, plane-specific lesion details.
• The 3D F-Res model uses a 128×128×128 patch size and processes volumetric data to model spatial context across slices.
• Fusion is performed via a weighted combination: $\text{softmax}_{\text{pred}} = 0.55 \times \text{softmax}_{3D} + 0.45 \times \text{softmax}_{2D}$, favoring the 3D output.
• A post-processing method is applied to further reduce false positives, particularly in high-uptake physiological regions.

Experimental results

Research questions

• RQ1 Can a joint 2D-3D nnUNet architecture improve whole-body lesion segmentation accuracy in 18F-FDG PET/CT compared to standalone 2D or 3D models?
• RQ2 Does integrating ResNet18 as a backbone in the 3D nnUNet improve performance over the standard UNet backbone in lesion segmentation?
• RQ3 How does weighted fusion of 2D and 3D predictions affect the balance between false positive and false negative rates?
• RQ4 Can cascading low-resolution predictions with high-resolution models improve segmentation performance, or does it degrade due to resolution loss?
• RQ5 To what extent can post-processing reduce false positives in regions of physiological FDG uptake such as the intestine and bladder?

Key findings

• The joint 2D-3D model achieved the highest overall ranking (rank 1) on the AutoPet Challenge validation set, outperforming all other models in Dice score, false positive volume, and false negative volume.
• The joint model achieved a Dice score of 0.79 on the total cohort, with 0.83 for lung cancer, 0.78 for melanoma, and 0.74 for lymphoma.
• The 3D F-Res model had the lowest false negative volume (1.93) across all disease types, indicating superior lesion detection capability.
• The 3D L-Res model had the lowest false positive volume (0.51) but the highest false negative volume (9.93), indicating a trade-off between sensitivity and specificity.
• The 2D UNet model showed the lowest false positive volume (1.63) but the highest false negative volume (4.60), highlighting its sensitivity to false positives in non-lesion regions.
• The joint model successfully mitigated false positives in melanoma and negative cases and compensated for missed lesions in lung cancer, demonstrating complementary strengths of 2D and 3D branches.