[Paper Review] CAE-Transformer: Transformer-based Model to Predict Invasiveness of Lung Adenocarcinoma Subsolid Nodules from Non-thin Section 3D CT Scans

Shahin Heidarian, Parnian Afshar+3 more|arXiv (Cornell University)|Oct 17, 2021

Lung Cancer Diagnosis and Treatment4 citations

TL;DR

This paper proposes CAE-Transformer, a transformer-based model that predicts the invasiveness of lung adenocarcinoma subsolid nodules using non-thin section 3D CT scans. It employs a convolutional autoencoder to extract features from 2D CT slices, which are then fed into a modified transformer to capture global inter-slice dependencies, achieving 87.73% accuracy, 88.67% sensitivity, 86.33% specificity, and 0.913 AUC on a 114-patient dataset with minimal radiologist supervision.

ABSTRACT

Lung cancer is the leading cause of mortality from cancer worldwide and has various histologic types, among which Lung Adenocarcinoma (LUAC) has recently been the most prevalent one. The current approach to determine the invasiveness of LUACs is surgical resection, which is not a viable solution to fight lung cancer in a timely fashion. An alternative approach is to analyze chest Computed Tomography (CT) scans. The radiologists' analysis based on CT images, however, is subjective and might result in a low accuracy. In this paper, a transformer-based framework, referred to as the "CAE-Transformer", is developed to efficiently classify LUACs using whole CT images instead of finely annotated nodules. The proposed CAE-Transformer can achieve high accuracy over a small dataset and requires minor supervision from radiologists. The CAE Transformer utilizes an encoder to automatically extract informative features from CT slices, which are then fed to a modified transformer to capture global inter-slice relations and provide classification labels. Experimental results on our in-house dataset of 114 pathologically proven Sub-Solid Nodules (SSNs) demonstrate the superiority of the CAE-Transformer over its counterparts, achieving an accuracy of 87.73%, sensitivity of 88.67%, specificity of 86.33%, and AUC of 0.913, using a 10-fold cross-validation.

Motivation & Objective

• To develop an automated, low-supervision framework for predicting the invasiveness of subsolid lung adenocarcinoma nodules from routine, non-thin-section 3D CT scans.
• To overcome the limitations of subjective radiologist interpretation and radiomics-based models that rely on manually annotated nodules.
• To leverage deep learning, particularly transformers, to model global inter-slice relationships in volumetric CT data without requiring thin-slice or high-resolution scans.
• To create a clinically applicable tool that reduces unnecessary surgeries by enabling timely, accurate preoperative classification of pre-invasive, minimally invasive, and invasive subtypes.
• To demonstrate that a self-supervised feature extraction approach (CAE) combined with a transformer encoder outperforms conventional models on small, real-world datasets.

Proposed method

• A Convolutional Autoencoder (CAE) is pre-trained in an unsupervised manner on 2D CT slices to extract hierarchical, informative features.
• The CAE-generated features from all slices of a nodule are stacked into a sequence of shape (25, 256), representing the full 3D nodule as a sequence of 2D feature maps.
• A modified transformer encoder processes the sequence to model long-range dependencies and inter-slice relationships using multi-head self-attention mechanisms.
• The final classification head uses a learned [CLS] token representation from the transformer to predict the invasiveness class (pre-invasive/minimally invasive vs. invasive).
• The model is fine-tuned using a low learning rate (1e-6) on the full dataset with label smoothing (α=0.05) to improve generalization.
• Performance is evaluated via 10-fold cross-validation, and ablation studies compare CAE-Transformer against models using global average pooling (GAP), global max pooling (GMP), LSTM, 3D-CNN, and ResNet variants.

Experimental results

Research questions

• RQ1 Can a transformer-based model effectively capture inter-slice relationships in 3D lung nodules from non-thin-section CT scans without requiring high-resolution or thin-slice imaging?
• RQ2 Does combining unsupervised feature learning via a convolutional autoencoder with a transformer encoder improve classification performance on small, real-world medical datasets?
• RQ3 How does the CAE-Transformer compare to traditional radiomics and deep learning baselines (e.g., 3D-CNN, LSTM, ResNet) in predicting the invasiveness of subsolid nodules?
• RQ4 To what extent does the model’s performance depend on the choice of global pooling strategy (GAP vs. GMP) when aggregating features before classification?
• RQ5 Can a self-supervised pre-training approach reduce the need for extensive radiologist annotation while maintaining high diagnostic accuracy?

Key findings

• The CAE-Transformer achieved 87.73% accuracy, 88.67% sensitivity, 86.33% specificity, and 0.913 AUC on a 114-patient dataset using 10-fold cross-validation.
• The model significantly outperformed the original radiomics-based model from Reference [7], which achieved 81.0% accuracy on the same dataset using manually annotated nodules.
• The CAE-Transformer surpassed non-transformer baselines, including GMP-FC, GAP-FC, CAE-LSTM, 3D-CNN, and ResNet-based models, on the same dataset.
• The ablation study showed that replacing global pooling (GMP/GAP) with a transformer encoder led to a substantial performance gain, confirming the importance of modeling long-range dependencies across slices.
• The model demonstrated robustness on small datasets, with deeper or larger networks (e.g., deeper ResNets) underperforming due to overfitting, highlighting the benefit of the CAE pre-training and attention mechanism.
• The use of label smoothing (α=0.05) improved generalization, indicating its value in small-data medical learning settings.