[Paper Review] Predicting Language Recovery after Stroke with Convolutional Networks on Stitched MRI

Yusuf Roohani, Noor Sajid+3 more|arXiv (Cornell University)|Nov 26, 2018

Acute Ischemic Stroke ManagementMedicine13 references3 citations

TL;DR

This paper proposes a novel deep learning approach that uses stitched 2D cross-sections of raw MRI scans in a 2D convolutional neural network (CNN) to predict language recovery after stroke, achieving performance comparable to models using expert-annotated lesion information. The method learns discriminative brain features that are complementary to lesion data, and saliency maps confirm activation in regions consistent with known language networks.

ABSTRACT

One third of stroke survivors have language difficulties. Emerging evidence suggests that their likelihood of recovery depends mainly on the damage to language centers. Thus previous research for predicting language recovery post-stroke has focused on identifying damaged regions of the brain. In this paper, we introduce a novel method where we only make use of stitched 2-dimensional cross-sections of raw MRI scans in a deep convolutional neural network setup to predict language recovery post-stroke. Our results show: a) the proposed model that only uses MRI scans has comparable performance to models that are dependent on lesion specific information; b) the features learned by our model are complementary to the lesion specific information and the combination of both appear to outperform previously reported results in similar settings. We further analyse the CNN model for understanding regions in brain that are responsible for arriving at these predictions using gradient based saliency maps. Our findings are in line with previous lesion studies.

Motivation & Objective

• To develop a deep learning method for predicting language recovery after stroke that relies solely on raw MRI scans, avoiding reliance on expert-annotated lesion data.
• To evaluate whether raw image features extracted from stitched 2D MRI slices can match or exceed the performance of models using lesion-specific features.
• To interpret the model’s predictions using gradient-based saliency maps and validate their alignment with known neuroanatomy of language.
• To assess the correlation between the CNN-learned features and manually extracted white/gray matter features to evaluate representational quality.

Proposed method

• Stitched 2D MRI slices are created by concatenating axial cross-sections of 3D MRI scans in anatomical order, preserving spatial correspondence across patients.
• A 2D convolutional neural network is trained on these stitched images to extract hierarchical features without prior lesion segmentation.
• A separate feed-forward neural network regresses the predicted language outcome (CAT spoken picture description score) using the 64-dimensional CNN-embedded features and demographic data.
• Gradient-based saliency maps are generated to visualize brain regions most influential in predictions, with results visualized in 3D anatomical planes.
• Distance correlation is computed between CNN-learned features and manually segmented white/gray matter to assess representational similarity.
• The model is trained and evaluated using the PLORAS dataset, with performance compared to prior lesion-based approaches.

Experimental results

Research questions

• RQ1 Can a deep CNN trained on stitched 2D MRI slices predict post-stroke language recovery with performance comparable to models using expert-annotated lesion information?
• RQ2 Are the features learned by the CNN complementary to lesion-specific features, such that their combination improves prediction accuracy?
• RQ3 Do the saliency maps generated by the model highlight brain regions known to be involved in language processing, such as the right prefrontal cortex?
• RQ4 How well do the CNN-learned features correlate with manually extracted white and gray matter features in terms of representation quality?

Key findings

• The proposed model achieved an R-squared of 0.75 and Pearson correlation of 0.78 when combining image and demographic features, outperforming previous state-of-the-art models that used lesion-specific features.
• The model's performance using only raw MRI data (R² = 0.56, Pearson r = 0.60) was comparable to models that relied on expert-annotated lesion information (R² = 0.50, Pearson r = 0.50).
• Saliency maps revealed significant activation in the right prefrontal region, consistent with prior neuroimaging studies on language recovery, validating the model’s biological plausibility.
• The distance correlation between CNN-learned features and manually segmented white/gray matter features was 0.7970, indicating strong representational alignment.
• The combination of CNN-derived image features and lesion-specific information yielded higher performance than either modality alone, demonstrating complementarity.