[Paper Review] CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing
CoRe-BT introduces a clinically grounded multimodal benchmark integrating MRI, whole-slide pathology, and diagnostic text, with a fusion framework (CoRe-BT-Fusion) to handle missing modalities at inference.
Accurate brain tumor typing requires integrating heterogeneous clinical evidence, including magnetic resonance imaging (MRI), histopathology, and pathology reports, which are often incomplete at the time of diagnosis. We introduce CoRe-BT, a cross-modal radiology-pathology-text benchmark for brain tumor typing, designed to study robust multimodal learning under missing modality conditions. The dataset comprises 310 patients with multi-sequence brain MRI (T1, T1c, T2, FLAIR), including 95 cases with paired H&E-stained whole-slide pathology images and pathology reports. All cases are annotated with tumor type and grade, and MRI volumes include expert-annotated tumor masks, enabling both region-aware modeling and auxiliary learning tasks. Tumors are categorized into six clinically relevant classes capturing the heterogeneity of common and rare glioma subtypes. We evaluate tumor typing under variable modality availability by comparing MRI-only models with multimodal approaches that incorporate pathology information when present. Baseline experiments demonstrate the feasibility of multimodal fusion and highlight complementary modality contributions across clinically relevant typing tasks. CoRe-BT provides a grounded testbed for advancing multimodal glioma typing and representation learning in realistic scenarios with incomplete clinical data.
Motivation & Objective
- Motivate robust brain tumor typing with incomplete multimodal clinical data.
- Create a clinically meaningful, pathologist-validated hierarchical labeling scheme for gliomas.
- Propose CoRe-BT-Fusion to fuse MRI and histopathology embeddings with text-aware context.
- Evaluate modality-availability scenarios and provide baseline multimodal results.
Proposed method
- Formalize a three-modality task with MRI, WSI, and pathology report inputs and variable observation sets at inference time.
- Extract MRI embeddings via NeuroVFM from 3D volumes and compute subject-level representations.
- Extract WSI embeddings via Prov-GigaPath to obtain slide-level representations with ultra-long context modeling.
- Fuse modality embeddings through a weighted probe scheme with a learnable residual gate to enable modality-aware predictions.
- Train linear probes per modality and combine them with a fusion module whose weights reflect modality relevance per subject.
- Ground truth labels follow a pathologist-validated hierarchical scheme; evaluation focuses on Level 1 categories and subtype-rich tasks.
Experimental results
Research questions
- RQ1Can multimodal fusion improve brain tumor typing under missing modalities compared to unimodal baselines?
- RQ2How does modality-ablated performance vary across Level 1 class, WHO grade, and LGG/HGG tasks?
- RQ3Does including pathology information (when available) improve fine-grained glioma typing beyond radiology alone?
Key findings
- Multimodal training with CoRe-BT-Fusion improves macro-accuracy, precision, recall, and F1-score over modality-expert linear probes.
- In LGG/HGG and WHO Grade tasks, pathology-ablated fusion sometimes outperforms the full fusion, suggesting the model benefits from pathology-informed interpretation of histology.
- For Level 1 classification, the multimodal CoRe-BT-Fusion model outperforms all variants, showing the greatest benefit for coarse-to-fine clinically meaningful categories.
- The benchmark demonstrates complementary contributions from radiology and pathology representations across clinically relevant typing tasks.
Better researchstarts right now
From reading papers to final review, dramatically reduce your research time.
No credit card · Free plan available
This review was created by AI and reviewed by human editors.