[Paper Review] A Normalized Fully Convolutional Approach to Head and Neck Cancer Outcome Prediction

William Le, Francisco Romero+1 more|arXiv (Cornell University)|May 28, 2020

Radiomics and Machine Learning in Medical Imaging3 references4 citations

TL;DR

This paper proposes a normalized fully convolutional network (FCN) preprocessor combined with an aggregated residual CNN (AggResCNN) for head and neck cancer survival prediction using pre-treatment PET-CT scans. By integrating multi-modal imaging data and leveraging feature normalization via FCN, the method achieves a 0.76 AUC—6 percentage points higher than state-of-the-art models—while reducing model parameters by over 27%, demonstrating improved performance and efficiency in outcome prediction.

ABSTRACT

In medical imaging, radiological scans of different modalities serve to enhance different sets of features for clinical diagnosis and treatment planning. This variety enriches the source information that could be used for outcome prediction. Deep learning methods are particularly well-suited for feature extraction from high-dimensional inputs such as images. In this work, we apply a CNN classification network augmented with a FCN preprocessor sub-network to a public TCIA head and neck cancer dataset. The training goal is survival prediction of radiotherapy cases based on pre-treatment FDG PET-CT scans, acquired across 4 different hospitals. We show that the preprocessor sub-network in conjunction with aggregated residual connection leads to improvements over state-of-the-art results when combining both CT and PET input images.

Motivation & Objective

• To improve survival outcome prediction in head and neck cancer patients using multi-modal pre-treatment PET-CT imaging.
• To address the challenge of high-dimensional, multi-domain medical image data by leveraging deep learning for robust feature extraction.
• To reduce model complexity and parameter count while maintaining or improving classification performance.
• To investigate whether a fully convolutional preprocessor can normalize and enhance features across PET and CT modalities for improved survival classification.
• To validate the method on a multi-institutional TCIA dataset with imbalanced survival outcomes.

Proposed method

• A fully convolutional network (FCN) sub-network with 4 downsampling and 4 upsampling blocks is used as a preprocessor to normalize multi-modal PET-CT inputs.
• The FCN employs 3x3 convolutions with Scaled Exponential Linear Unit (SeLU) activation and skip connections between corresponding downsampling and upsampling blocks.
• An 18-layer aggregated residual CNN (AggResCNN) is used as the classifier, incorporating grouped convolutions (cardinality 32) and residual shortcuts to reduce vanishing gradients.
• The FCN output is concatenated with the CT image and fed into the AggResCNN, which uses global average pooling and a softmax layer for binary classification (survival vs. death).
• Data augmentation (flips, shifts, rotations) is applied to rebalance the dataset, which contains only 19% death cases (56/298 patients).
• The model is trained end-to-end using categorical cross-entropy loss with Adam optimizer (initial LR 0.0006) for 100 epochs on a single GPU.

Experimental results

Research questions

• RQ1 Can a fully convolutional preprocessor improve feature representation for multi-modal PET-CT imaging in head and neck cancer survival prediction?
• RQ2 Does combining PET and CT images through a deep learning architecture lead to better survival prediction than using CT alone?
• RQ3 Can aggregated residual convolutions reduce model parameters while maintaining or improving performance compared to prior state-of-the-art models?
• RQ4 Does the FCN preprocessor enhance performance by learning features beyond the primary tumor volume (GTV)?
• RQ5 How does the inclusion of PET images affect classification performance relative to CT-only inputs?

Key findings

• The proposed FCN+AggResCNN model achieves an AUC of 0.76 on the TCIA head and neck cancer dataset, representing a 6-percentage-point improvement over the prior state-of-the-art (0.70).
• The model reduces parameter count to 683,650, a 27% reduction compared to the prior state-of-the-art model (930,146 parameters).
• The FCN preprocessor improves performance on PET+CT inputs (AUC 0.76) compared to the base AggResCNN (AUC 0.74), indicating effective multi-modal feature normalization.
• Performance on masked CT (tumor region only) is better than on full CT for the CNN baseline (0.67 vs. 0.57), but the FCN+AggResCNN model performs better on full CT (0.70) than on masked CT (0.67), suggesting the FCN captures useful non-GTV features.
• PET-only input yields the worst performance (AUC 0.57 for FCN+AggResCNN), indicating PET contributes limited independent predictive power due to low resolution and lack of uptake in non-tumoral tissues.
• The model achieves 91% sensitivity and 61% specificity, indicating strong detection of death cases despite class imbalance.