[Paper Review] Fibro-CoSANet: Pulmonary Fibrosis Prognosis Prediction using a Convolutional Self Attention Network

Zabir Al Nazi, Fazla Rabbi Mashrur+2 more|arXiv (Cornell University)|Apr 13, 2021

Interstitial Lung Diseases and Idiopathic Pulmonary Fibrosis27 references26 citations

TL;DR

Fibro-CoSANet is a novel end-to-end multi-modal convolutional self-attention network that predicts pulmonary fibrosis prognosis by fusing chest CT images and demographic data. It achieves a new state-of-the-art modified Laplace Log-Likelihood score of -6.68 on the OSIC dataset by using a single CT slice, lung volume estimation, and attention-enhanced feature refinement to predict FVC decline slope with high efficiency and accuracy.

ABSTRACT

Idiopathic pulmonary fibrosis (IPF) is a restrictive interstitial lung disease that causes lung function decline by lung tissue scarring. Although lung function decline is assessed by the forced vital capacity (FVC), determining the accurate progression of IPF remains a challenge. To address this challenge, we proposed Fibro-CoSANet, a novel end-to-end multi-modal learning-based approach, to predict the FVC decline. Fibro-CoSANet utilized CT images and demographic information in convolutional neural network frameworks with a stacked attention layer. Extensive experiments on the OSIC Pulmonary Fibrosis Progression Dataset demonstrated the superiority of our proposed Fibro-CoSANet by achieving the new state-of-the-art modified Laplace Log-Likelihood score of -6.68. This network may benefit research areas concerned with designing networks to improve the prognostic accuracy of IPF. The source-code for Fibro-CoSANet is available at: \url{https://github.com/zabir-nabil/Fibro-CoSANet}.

Motivation & Objective

• To improve the accuracy of idiopathic pulmonary fibrosis (IPF) prognosis prediction using medical imaging and clinical data.
• To address the challenge of variable IPF progression by developing a robust, end-to-end learning framework.
• To enhance feature representation in CT images by integrating a stacked self-attention mechanism on top of convolutional features.
• To reduce computational overhead by modeling FVC decline as a linear trend prediction rather than full trajectory regression.
• To demonstrate the effectiveness of multi-modal fusion between deep imaging features and shallow demographic features for IPF prognosis.

Proposed method

• Fibro-CoSANet uses a single randomly selected CT slice per patient as input to a convolutional neural network (CNN) backbone.
• It estimates lung volume from all available CT scans to serve as a shallow, global feature for multi-modal fusion.
• Demographic features (age, sex, smoking status) are normalized and concatenated with the CNN-processed CT features.
• A stacked self-attention layer is applied on the fused features to refine attention to relevant lung regions and improve representation.
• The network predicts the slope of the FVC trend using a linear prior assumption, reducing complexity compared to full regression.
• The model is trained using mean squared error loss with pseudo-labels derived from SVD-based FVC slope computation.

Experimental results

Research questions

• RQ1 Can a single CT slice combined with demographic data and self-attention improve FVC progression prediction compared to full-scan or multi-slice approaches?
• RQ2 Does stacking self-attention layers on top of CNN features enhance performance in predicting IPF prognosis?
• RQ3 Can a linear prior assumption on FVC slope achieve better or comparable results than complex regression models with lower computational cost?
• RQ4 How does multi-modal fusion of imaging and demographic data affect model generalization and robustness in IPF prognosis?
• RQ5 What is the optimal balance between model complexity and performance in low-data regimes like the OSIC dataset?

Key findings

• Fibro-CoSANet achieved a new state-of-the-art modified Laplace Log-Likelihood score of -6.68, outperforming prior methods on the OSIC Pulmonary Fibrosis Progression dataset.
• The EfficientNet-B2 backbone with a stacked self-attention module achieved the best performance with a score of -6.68 ± 0.31 and RMSE of 181.5 ± 25.88.
• ResNet-50 and EfficientNet-B2 with attention layers achieved superior results compared to deeper ResNets, suggesting overfitting risks in smaller datasets.
• The use of a single CT slice with lung volume estimation reduced inference time and computational cost while maintaining high accuracy.
• EfficientNet-B0 showed the lowest computational complexity (0.07 GMacs, 4.05M parameters) but underperformed, indicating that model capacity must be balanced with data size.
• The addition of the self-attention module improved performance across most backbones, especially for EfficientNet-B1, B2, B3, B4, and ResNet-50, confirming its value in feature refinement.