[Paper Review] Merging multiple input descriptors and supervisors in a deep neural network for tractogram filtering

Daniel Jörgens, Pierre‐Marc Jodoin+2 more|arXiv (Cornell University)|Jul 11, 2023

Advanced Neuroimaging Techniques and ApplicationsMedicine3 citations

TL;DR

This paper proposes a deep learning framework that fuses multiple tractogram filtering strategies—TractQuerier, RecobundlesX, TractSeg, and an anatomy-inspired filter—into a single neural network to classify streamlines as plausible, implausible, or inconclusive. The model achieves high accuracy by leveraging streamline coordinates and diffusion data as the most informative features, demonstrating that geometric and microstructural information are critical for robust tractogram filtering.

ABSTRACT

One of the main issues of the current tractography methods is their high false-positive rate. Tractogram filtering is an option to remove false-positive streamlines from tractography data in a post-processing step. In this paper, we train a deep neural network for filtering tractography data in which every streamline of a tractogram is classified as {\em plausible, implausible}, or {\em inconclusive}. For this, we use four different tractogram filtering strategies as supervisors: TractQuerier, RecobundlesX, TractSeg, and an anatomy-inspired filter. Their outputs are combined to obtain the classification labels for the streamlines. We assessed the importance of different types of information along the streamlines for performing this classification task, including the coordinates of the streamlines, diffusion data, landmarks, T1-weighted information, and a brain parcellation. We found that the streamline coordinates are the most relevant followed by the diffusion data in this particular classification task.

Motivation & Objective

• To address the high false-positive rate in tractography by developing a unified filtering approach that classifies streamlines into three categories: plausible, implausible, and inconclusive.
• To integrate multiple tractogram filtering strategies as supervisory signals to improve classification robustness and generalization.
• To assess the relative importance of different input descriptors—such as streamline coordinates, diffusion data, T1-weighted imaging, landmarks, and parcellation—for accurate filtering.
• To evaluate whether combining multiple supervisory signals enhances filtering performance beyond individual methods.

Proposed method

• The method trains a deep neural network using four distinct filtering strategies as supervisory signals: TractQuerier, RecobundlesX, TractSeg, and an anatomy-inspired filter.
• Input features include 3D streamline coordinates, diffusion MRI data along the streamline, T1-weighted MRI intensity, brain landmarks, and a parcellation map.
• The model performs end-to-end classification of each streamline into one of three classes: plausible, implausible, or inconclusive.
• Ablation studies are conducted to evaluate the contribution of each input descriptor by training models with and without individual features.
• The network is trained using standard deep learning optimization with cross-entropy loss and evaluated using accuracy metrics across all four supervisors.
• Normalization of streamline coordinates to [-1,1] is applied, with future work considering MNI template-based alignment for improved performance.

Figure 1: Overview of streamline classification approach. An individual streamline is resampled before five streamline- and image-based descriptors are extracted along its extent: the coordinates of the streamline (xyz), T1-weighted information, spherical harmonics coefficients of the diffusion data

Experimental results

Research questions

• RQ1 Can a deep neural network effectively learn to classify streamlines into plausible, implausible, and inconclusive categories by leveraging multiple filtering strategies as supervisory signals?
• RQ2 Which input descriptors—coordinates, diffusion data, T1-weighted imaging, landmarks, or parcellation—are most predictive for accurate streamline classification?
• RQ3 How does the performance of the model change when individual input features are removed, and what does this reveal about feature redundancy or complementarity?
• RQ4 To what extent does the model's performance depend on geometric information (e.g., streamline coordinates) versus microstructural data (e.g., diffusion metrics)?

Key findings

• The 3D coordinates of streamlines are the most informative input descriptor for tractogram filtering, followed by diffusion data along the streamline.
• The model achieves high classification accuracy by combining multiple supervisory signals, outperforming models trained on individual inputs.
• When trained without streamline coordinates, the model still performs better than any single-input model, indicating complementary information across features.
• The model successfully extracts predictive information from all input types, including diffusion data and T1-weighted intensity, though their individual contributions are less dominant than coordinates.
• The ablation study reveals that no other input can substitute for the information contained in streamline coordinates, highlighting their irreplaceable role.
• The results suggest that future improvements may come from pre-aligning tractograms to the MNI template, which could further enhance performance.

Figure 2: Average number of streamlines per label and supervisor method. The number of streamlines for the positive and negative labels obtained from each of the supervisor methods is averaged over all 20 subjects. The error bars indicate the standard deviation for the particular groups.