[论文解读] FedNER: Privacy-preserving Medical Named Entity Recognition with Federated Learning

Suyu Ge, Fangzhao Wu+4 more|arXiv (Cornell University)|Mar 20, 2020

Topic Modeling参考文献 36被引用 46

一句话总结

FedNER 在多个人隐私敏感平台上训练医疗命名实体识别（NER）模型，通过将模型分解为共享模块和私有模块，并在联邦环境中仅聚合共享模块梯度。

ABSTRACT

Medical named entity recognition (NER) has wide applications in intelligent healthcare. Sufficient labeled data is critical for training accurate medical NER model. However, the labeled data in a single medical platform is usually limited. Although labeled datasets may exist in many different medical platforms, they cannot be directly shared since medical data is highly privacy-sensitive. In this paper, we propose a privacy-preserving medical NER method based on federated learning, which can leverage the labeled data in different platforms to boost the training of medical NER model and remove the need of exchanging raw data among different platforms. Since the labeled data in different platforms usually has some differences in entity type and annotation criteria, instead of constraining different platforms to share the same model, we decompose the medical NER model in each platform into a shared module and a private module. The private module is used to capture the characteristics of the local data in each platform, and is updated using local labeled data. The shared module is learned across different medical platform to capture the shared NER knowledge. Its local gradients from different platforms are aggregated to update the global shared module, which is further delivered to each platform to update their local shared modules. Experiments on three publicly available datasets validate the effectiveness of our method.

研究动机与目标

• 说明在每个平台上标注数据有限且存在隐私约束时为何需要医疗 NER。
• 提出 FedNER，以在不交换原始数据的情况下利用跨平台标注数据。
• 展示将模型分解为共享和私有组件如何改善跨平台学习。
• 展示通过中央服务器仅聚合共享模块梯度来实现隐私保护的训练。

提出的方法

• 将医疗 NER 表述为以词、字符和语言模型嵌入为特征的序列标注。
• 构建一个三部分的 NER 模型：词表示、上下文建模（CNN + Bi-LSTM）和 CRF 解码。
• 将模型分解为共享模块（底部层和嵌入）和私有模块（顶层：Bi-LSTM 和 CRF）。
• 在本地数据上对私有模块进行私有化训练；将共享模块的梯度发送到中央服务器。
• 服务器使用加权求和聚合跨平台的共享模块梯度，并更新全局共享模块，然后重新分发。
• 重复若干轮直到收敛。
• 在三个公开数据集上使用 BIO 标记，并以严格和放松的 F1 进行评估。

实验结果

研究问题

• RQ1 FedNER 是否能够通过在不共享原始数据的情况下利用来自多个隐私敏感平台的数据来提升医疗 NER？
• RQ2 将模型分解为共享和私有模块是否比完全共享或完全私有的做法获得更好的性能？
• RQ3 在具有非 IID 数据分布的多样化医疗 NER 数据集上，FedNER 的表现如何？

主要发现

• FedNER 在三个医疗 NER 数据集上持续优于单平台训练和若干基线。
• 将模型分解为共享和私有模块在保留平台特征的同时有效捕获跨平台知识。
• 当跨平台的重叠实体信息增加且每个平台的数据较少时，FedNER 的性能提升更明显。
• FedNER 是一个通用框架，在 FedNER 框架下应用可以改进不同的医疗 NER 方法。