[Paper Review] Large-Scale Unsupervised Deep Representation Learning for Brain Structure

Ayush Jaiswal, Dong Guo+2 more|arXiv (Cornell University)|May 2, 2018

Functional Brain Connectivity Studies10 references3 citations

TL;DR

This paper proposes a large-scale unsupervised deep learning framework using convolutional autoencoders (CAEs) to learn generic, low-dimensional representations of structural brain MRI scans without domain-specific features or manual intervention. The method achieves classification performance comparable to FreeSurfer features—on par with state-of-the-art hand-crafted features—while reducing feature extraction time from hours to under half a second per scan.

ABSTRACT

Machine Learning (ML) is increasingly being used for computer aided diagnosis of brain related disorders based on structural magnetic resonance imaging (MRI) data. Most of such work employs biologically and medically meaningful hand-crafted features calculated from different regions of the brain. The construction of such highly specialized features requires a considerable amount of time, manual oversight and careful quality control to ensure the absence of errors in the computational process. Recent advances in Deep Representation Learning have shown great promise in extracting highly non-linear and information-rich features from data. In this paper, we present a novel large-scale deep unsupervised approach to learn generic feature representations of structural brain MRI scans, which requires no specialized domain knowledge or manual intervention. Our method produces low-dimensional representations of brain structure, which can be used to reconstruct brain images with very low error and exhibit performance comparable to FreeSurfer features on various classification tasks.

Motivation & Objective

• To develop an unsupervised deep representation learning method for structural brain MRI that eliminates reliance on hand-crafted, domain-specific features.
• To address the limitations of traditional tools like FreeSurfer, which require manual quality control and assume fixed anatomical priors that fail in abnormal brains.
• To create a scalable, end-to-end deep learning framework that generalizes across diverse brain imaging datasets and adapts with more data.
• To evaluate whether self-supervised features from CAEs can match or exceed the performance of established FreeSurfer features in clinical classification tasks.

Proposed method

• Trained three CAE architectures—CAE-staged, CAE-joint, and CAE-3D—on 15,485 sBMRI scans from nine public datasets, including ADNI, ABIDE-I, and HCP.
• CAE-staged uses two separate autoencoders: one for 2D axial slices and another to combine frame-level representations into a 3D brain embedding.
• CAE-joint merges the two stages into a single end-to-end trainable network for joint feature learning across 2D slices.
• CAE-3D applies 3D convolutions directly to volumetric MRI data to learn 3D latent representations without slice-wise processing.
• Applied skull-stripping and cropping to 200×200×200 isotropic voxels to standardize input data across all sources.
• Used reconstruction loss and latent space regularization to ensure meaningful, low-dimensional representations that preserve structural information.

Experimental results

Research questions

• RQ1 Can unsupervised deep representation learning via CAEs produce brain MRI features that match the diagnostic performance of FreeSurfer’s hand-crafted features?
• RQ2 How does the performance of CAE-based features compare to FreeSurfer features across multiple classification tasks involving neurodegenerative and neurodevelopmental disorders?
• RQ3 To what extent do the proposed CAE models reduce feature extraction time compared to FreeSurfer’s computationally intensive pipeline?
• RQ4 Does combining data from nine diverse neuroimaging datasets improve the robustness and generalization of the learned representations?
• RQ5 Can the learned representations generalize across abnormal brain morphologies where traditional anatomical priors fail?

Key findings

• CAE-3D achieved AUROC scores of 0.81 on H-ADNI vs. AD and 0.80 on H-ABIDE vs. ASD using logistic regression, matching or exceeding FreeSurfer’s performance on these tasks.
• CAE-joint and CAE-3D outperformed CAE-staged on most classification tasks, with CAE-3D showing the strongest generalization across datasets.
• Feature extraction using CAE-3D took only 0.45 seconds per scan, compared to 20–47 hours per scan using FreeSurfer’s ReconAll pipeline.
• The CAE-based features reconstructed brain images with very low reconstruction error, indicating high information retention in the latent space.
• On the H-ADNI vs. MCI task, CAE-3D achieved AUROC of 0.76 with logistic regression and 0.80 with random forest, closely matching FreeSurfer’s 0.77 and 0.81 respectively.
• The method demonstrated robustness across diverse datasets, including those with abnormal brain structures, without relying on fixed anatomical priors.