[Paper Review] FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning

Suyu Ge, Fangzhao Wu+4 more|arXiv (Cornell University)|Mar 20, 2020

Topic Modeling36 references46 citations

TL;DR

FedNER trains medical NER models across multiple privacy-sensitive platforms by decomposing models into shared and private modules and aggregating only shared-module gradients in a federated setting.

ABSTRACT

Medical named entity recognition (NER) has wide applications in intelligent healthcare. Sufficient labeled data is critical for training accurate medical NER model. However, the labeled data in a single medical platform is usually limited. Although labeled datasets may exist in many different medical platforms, they cannot be directly shared since medical data is highly privacy-sensitive. In this paper, we propose a privacy-preserving medical NER method based on federated learning, which can leverage the labeled data in different platforms to boost the training of medical NER model and remove the need of exchanging raw data among different platforms. Since the labeled data in different platforms usually has some differences in entity type and annotation criteria, instead of constraining different platforms to share the same model, we decompose the medical NER model in each platform into a shared module and a private module. The private module is used to capture the characteristics of the local data in each platform, and is updated using local labeled data. The shared module is learned across different medical platform to capture the shared NER knowledge. Its local gradients from different platforms are aggregated to update the global shared module, which is further delivered to each platform to update their local shared modules. Experiments on three publicly available datasets validate the effectiveness of our method.

Motivation & Objective

• Motivate the need for medical NER with limited labeled data on each platform and privacy constraints.
• Propose FedNER to leverage cross-platform labeled data without exchanging raw data.
• Show that decomposing models into shared and private components improves cross-platform learning.
• Demonstrate privacy-preserving training by aggregating only shared-module gradients through a central server.

Proposed method

• Formulate medical NER as sequence labeling with word, character, and language-model embeddings.
• Build a three-part NER model: word representation, context modeling (CNN + Bi-LSTM), and CRF decoding.
• Decompose the model into a shared module (bottom layers and embeddings) and a private module (top layers: Bi-LSTM and CRF).
• Train privately on local data for the private module; send gradients of the shared module to a central server.
• Server aggregates shared-module gradients across platforms using a weighted sum and updates the global shared module which is redistributed.
• Repeat across rounds until convergence.
• Use BIO tagging across three public datasets and evaluate with strict and relaxed F1.”

Experimental results

Research questions

• RQ1 Can FedNER improve medical NER by leveraging data from multiple privacy-sensitive platforms without sharing raw data?
• RQ2 Does decomposing the model into shared and private modules yield better performance than fully shared or fully private approaches?
• RQ3 How does FedNER perform across diverse medical NER datasets with non-IID data distributions?

Key findings

• FedNER consistently outperforms single-platform training and several baselines on three medical NER datasets.
• Decomposing into shared and private modules effectively captures cross-platform knowledge while preserving platform-specific characteristics.
• Performance gains from FedNER grow as the amount of overlapped entity information across platforms increases and when data per platform is scarce.
• FedNER is a general framework that can improve different medical NER methods when applied under the FedNER framework.