[Paper Review] Split learning for health: Distributed deep learning without sharing raw patient data

Praneeth Vepakomma, Otkrist Gupta+2 more|arXiv (Cornell University)|Dec 3, 2018

Machine Learning in HealthcareComputer Science346 citations

TL;DR

The paper presents SplitNN, a distributed deep learning framework that enables collaborative model training across health entities without sharing raw data or model details, and compares it to federated learning and large-batch SGD across multiple configurations.

ABSTRACT

Can health entities collaboratively train deep learning models without sharing sensitive raw data? This paper proposes several configurations of a distributed deep learning method called SplitNN to facilitate such collaborations. SplitNN does not share raw data or model details with collaborating institutions. The proposed configurations of splitNN cater to practical settings of i) entities holding different modalities of patient data, ii) centralized and local health entities collaborating on multiple tasks and iii) learning without sharing labels. We compare performance and resource efficiency trade-offs of splitNN and other distributed deep learning methods like federated learning, large batch synchronous stochastic gradient descent and show highly encouraging results for splitNN.

Motivation & Objective

• Motivate the need for privacy-preserving collaborative health data modeling under HIPAA and consent constraints.
• Propose SplitNN configurations suitable for multi-modal and multi-institution health data collaboration.
• Show resource efficiency advantages of SplitNN over federated learning and large-batch SGD.
• Demonstrate practical configurations for vertically partitioned data and label-sharing variants.

Proposed method

• Introduce simple vanilla SplitNN where each client trains up to a cut layer and sends activations to a server for rest of forward/backward passes.
• Present U-shaped configurations that avoid label sharing while using server-side layers to complete training.
• Describe vertically partitioned data configuration where different institutions hold different modalities and concatenate cut-layer outputs at a server.
• Compare SplitNN against federated learning and large-batch SGD in terms of validation accuracy, client FLOPs, and communication bandwidth.
• Highlight that computation is split at cut layers, reducing client-side workload while preserving data privacy.

Experimental results

Research questions

• RQ1 Can SplitNN enable effective multi-institution health data collaboration without sharing raw data or labels?
• RQ2 How do SplitNN configurations compare to federated learning and large-batch SGD in accuracy, computation, and bandwidth?
• RQ3 What are the resource-efficiency trade-offs for SplitNN in vertically partitioned and multi-modal health data settings?
• RQ4 Can SplitNN support label-less or privacy-sensitive configurations in real-world health scenarios?

Key findings

• SplitNN shows higher accuracies with dramatically lower client-side computation (e.g., 0.1548 TFlops vs 29.4 TFlops for 100 clients on CIFAR-10 with VGG).
• SplitNN requires less client bandwidth than large-batch SGD and federated learning at larger client counts (e.g., 1.2 GB vs 13 GB for 500 clients on CIFAR-100 with ResNet).
• Federated learning and large-batch SGD are more resource-heavy on the client side than SplitNN across the reported settings.
• SplitNN configurations support multi-modal and vertically partitioned data without sharing raw inputs or labels.
• The results indicate SplitNN can maintain accuracy while reducing computational burden and communication needs relative to baseline distributed methods.