[Paper Review] The Federated Tumor Segmentation (FeTS) Challenge

Sarthak Pati, Ujjwal Baid+30 more|arXiv (Cornell University)|May 12, 2021

Radiomics and Machine Learning in Medical ImagingMedicine45 references42 citations

TL;DR

Describes the FeTS 2021 federated learning challenge for brain tumor segmentation, with two tasks: federated weight aggregation and federated evaluation in the wild across multi-institution data without sharing patient data.

ABSTRACT

This manuscript describes the first challenge on Federated Learning, namely the Federated Tumor Segmentation (FeTS) challenge 2021. International challenges have become the standard for validation of biomedical image analysis methods. However, the actual performance of participating (even the winning) algorithms on "real-world" clinical data often remains unclear, as the data included in challenges are usually acquired in very controlled settings at few institutions. The seemingly obvious solution of just collecting increasingly more data from more institutions in such challenges does not scale well due to privacy and ownership hurdles. Towards alleviating these concerns, we are proposing the FeTS challenge 2021 to cater towards both the development and the evaluation of models for the segmentation of intrinsically heterogeneous (in appearance, shape, and histology) brain tumors, namely gliomas. Specifically, the FeTS 2021 challenge uses clinically acquired, multi-institutional magnetic resonance imaging (MRI) scans from the BraTS 2020 challenge, as well as from various remote independent institutions included in the collaborative network of a real-world federation (https://www.fets.ai/). The goals of the FeTS challenge are directly represented by the two included tasks: 1) the identification of the optimal weight aggregation approach towards the training of a consensus model that has gained knowledge via federated learning from multiple geographically distinct institutions, while their data are always retained within each institution, and 2) the federated evaluation of the generalizability of brain tumor segmentation models "in the wild", i.e. on data from institutional distributions that were not part of the training datasets.

Motivation & Objective

• Motivate and enable robust brain tumor segmentation using federated learning across geographically distinct institutions while preserving data privacy.
• Identify effective federated weight aggregation strategies for consensus model building without data pooling.
• Evaluate model generalizability under real-world distribution shifts via federated evaluation across institutions.
• Provide standardized datasets, pre-processing, and annotation protocols to support reproducible FL-based segmentation research.

Proposed method

• Use multi-institutional mpMRI data from BraTS 2020 and FeTS federation with ground-truth labels for tumor sub-regions.
• Apply a fixed U-Net architecture with residual connections to focus on aggregation method evaluation rather than segmentation architecture.
• Organize training as federated rounds where collaborators locally train and send updates to a central aggregator for consensus model creation.
• Assess aggregation strategies and communication efficiency, including optional handling of network outages (stragglers).
• Evaluate on federated test data retained at each institution, enabling in-wild generalization assessment across diverse domains.
• Measure segmentation quality with Dice similarity coefficient and robust 95th percentile Hausdorff distance; monitor communication budget as a round-based cost.

Experimental results

Research questions

• RQ1 What is the best approach to aggregate knowledge from locally trained models across institutions to form a robust consensus segmentation model?
• RQ2 How well do federated segmentation models generalize to data from institutions not represented in the training set (domain generalization in the wild)?
• RQ3 How do communication efficiency and handling of outages affect federated learning performance in this setting?

Key findings

• This manuscript presents the first Federated Learning brain tumor segmentation challenge and outlines two concrete tasks for FL weight aggregation and federated evaluation.
• The evaluation relies on Dice similarity coefficient and 95th percentile Hausdorff distance as primary segmentation metrics.
• A fixed U-Net with residual connections is used to isolate the impact of federation methodologies on performance.
• Pre-processing follows BraTS pipelines including atlas registration and brain extraction, with publicly available CaPTk and FeTS tools.
• Code and pipelines for reproducibility are released through CaPTk and FeTS repositories.
• The challenge framework emphasizes privacy-preserving, distributed evaluation where data remain at local sites.