[Paper Review] Extracting Universal Representations of Cognition across Brain-Imaging Studies

Arthur Mensch, Julien Mairal+2 more|arXiv (Cornell University)|Sep 17, 2018

Functional Brain Connectivity Studies55 references3 citations

TL;DR

This paper proposes a multi-task learning framework that extracts universal brain representations across 35 task-fMRI studies without requiring a unified cognitive theory. By jointly training a linear decoding model with shared parameters, it boosts statistical power and improves decoding accuracy by 80% of studies, revealing interpretable brain networks tuned to psychological manipulations.

ABSTRACT

Cognitive brain imaging is accumulating datasets about the neural substrate of many different mental processes. Yet, most studies are based on few subjects and have low statistical power. Analyzing data across studies could bring more statistical power; yet the current brain-imaging analytic framework cannot be used at scale as it requires casting all cognitive tasks in a unified theoretical framework. We introduce a new methodology to analyze brain responses across tasks without a joint model of the psychological processes. The method boosts statistical power in small studies with specific cognitive focus by analyzing them jointly with large studies that probe less focal mental processes. Our approach improves decoding performance for 80% of 35 widely-different functional-imaging studies. It finds commonalities across tasks in a data-driven way, via common brain representations that predict mental processes. These are brain networks tuned to psychological manipulations. They outline interpretable and plausible brain structures. The extracted networks have been made available; they can be readily reused in new neuro-imaging studies. We provide a multi-study decoding tool to adapt to new data.

Motivation & Objective

• Address the low statistical power in individual fMRI studies despite growing data availability.
• Overcome the limitation of standard meta-analyses, which require a unified cognitive theory and sacrifice specificity.
• Enable joint analysis of diverse fMRI studies with different task paradigms by extracting shared brain representations.
• Improve decoding performance in small, focused studies by leveraging large-scale, multi-task data.
• Develop reusable, interpretable brain networks that serve as universal priors for future neuroimaging studies.

Proposed method

• Formulate multi-task linear decoding where each study is a separate task with its own brain response mapping.
• Share model parameters across studies to discover commonalities in brain responses to psychological manipulations.
• Use non-convex optimization and regularization techniques to train the joint model on 35 diverse fMRI studies.
• Extract shared brain representations that predict mental processes across tasks, forming interpretable functional networks.
• Train the model on contrast maps from each study without requiring manual annotation or cross-study alignment.
• Make the learned representations publicly available for reuse in new studies and multi-study decoding tools.

Experimental results

Research questions

• RQ1 Can brain representations be extracted across diverse fMRI studies without a unified cognitive framework?
• RQ2 Does joint analysis of multiple fMRI studies improve decoding performance compared to individual study analysis?
• RQ3 Can shared representations from large-scale data enhance statistical power in small, focused studies?
• RQ4 Are the extracted brain networks interpretable and biologically plausible in terms of known functional anatomy?
• RQ5 To what extent can the learned representations generalize to new, unseen cognitive paradigms?

Key findings

• The method improved decoding performance in 80% of the 35 task-fMRI studies analyzed, with average accuracy gains across tasks.
• For the localizer task, decoding accuracy reached 99% for face vs. baseline and 100% for house vs. scramble, indicating high reliability.
• In the auditory compression task, the model achieved 99% accuracy for visual language vs. bottleneck, demonstrating robustness across sensory modalities.
• The approach enabled significant improvements in small studies, such as a 3% accuracy gain in the foreign language task (french vs. korean: 97% vs. 93%).
• The extracted brain representations were found to be interpretable, outlining plausible functional brain structures linked to specific psychological manipulations.
• The method outperformed single-study decoding by leveraging shared information across studies, even when tasks were not directly comparable.