[Paper Review] Split Learning for collaborative deep learning in healthcare

Maarten G. Poirot, Praneeth Vepakomma+4 more|arXiv (Cornell University)|Dec 27, 2019

Machine Learning in Healthcare23 references82 citations

TL;DR

Demonstrates that split learning enables collaborative deep learning across multiple healthcare centers, achieving comparable or superior performance to centralized or non-collaborative setups, across retinal and chest X-ray tasks, with benefits sustaining as the number of clients increases.

ABSTRACT

Shortage of labeled data has been holding the surge of deep learning in healthcare back, as sample sizes are often small, patient information cannot be shared openly, and multi-center collaborative studies are a burden to set up. Distributed machine learning methods promise to mitigate these problems. We argue for a split learning based approach and apply this distributed learning method for the first time in the medical field to compare performance against (1) centrally hosted and (2) non collaborative configurations for a range of participants. Two medical deep learning tasks are used to compare split learning to conventional single and multi center approaches: a binary classification problem of a data set of 9000 fundus photos, and multi-label classification problem of a data set of 156,535 chest X-rays. The several distributed learning setups are compared for a range of 1-50 distributed participants. Performance of the split learning configuration remained constant for any number of clients compared to a single center study, showing a marked difference compared to the non collaborative configuration after 2 clients (p < 0.001) for both sets. Our results affirm the benefits of collaborative training of deep neural networks in health care. Our work proves the significant benefit of distributed learning in healthcare, and paves the way for future real-world implementations.

Motivation & Objective

• Motivate and address data scarcity and privacy in healthcare deep learning.
• Propose a split learning (U-shaped) framework suitable for healthcare settings.
• Evaluate split learning against centralized and non-collaborative baselines on medical imaging tasks.
• Assess scalability across 1-50 distributed participants and analyze performance stability.

Proposed method

• Apply a U-shaped split learning configuration with three links (front local, center central, back local).
• Train networks sequentially across clients without sharing raw data or labels; transfer local state when switching clients.
• Use Resnet-34 for Diabetic Retinopathy and DenseNet-121 for CheXpert with standard Adam optimization.
• Compare performance to centralized training and non-collaborative training; evaluate with accuracy (DR) and AUROC (CheXpert).
• Partition data into 75% training and 25% validation, equal data split across clients, with no patient overlap.
• Use one epoch per client in collaborative mode; non-collaborative mode mirrors data size of the collaborative setting.

Experimental results

Research questions

• RQ1 Does split learning yield performance comparable to centralized training in medical imaging tasks?
• RQ2 How does collaboration via split learning scale with increasing numbers of distributed participants (1-50)?
• RQ3 What is the performance gap between split learning and non-collaborative training across tasks and client counts?
• RQ4 What are the privacy and bandwidth implications of the U-shaped split learning configuration in healthcare?
• RQ5 How does split learning performance compare to other distributed approaches (as referenced) in non-medical settings?

Key findings

• Split learning consistently outperformed non-collaborative training across both datasets.
• On CheXpert, mean performance in non-collaborative settings was significantly lower than collaborative settings for more than 2 clients (p<0.005).
• DR results show stable performance for split learning across 1-50 clients, with higher accuracy than non-collaborative configurations as client count increases.
• Table 1 reports DR results where, for 1 client, split learning = 0.888 vs non-collaborative = 0.869; for 2 clients, 0.850 vs 0.852; and for higher client counts, split learning remains competitive (values shown in table).
• CheXpert results use AUROC across five tasks; split learning beats non-collaborative configurations, with significant differences emerging as the number of clients grows.
• The study concludes that distributed training in healthcare via split learning yields notable performance gains and supports future real-world implementations.