[论文解读] BERT-based Ranking for Biomedical Entity Normalization

Zongcheng Ji, Qiang Wei+1 more|arXiv (Cornell University)|Aug 9, 2019

Biomedical Text Mining and Ontologies参考文献 32被引用 93

一句话总结

这篇论文对预训练的 BERT、BioBERT 和 ClinicalBERT 模型进行微调用于生物医学实体规范化，并在三个数据集上展示了最先进的准确性提升，相较于此前的方法，准确率最高提高至 1.17%。

ABSTRACT

Developing high-performance entity normalization algorithms that can alleviate the term variation problem is of great interest to the biomedical community. Although deep learning-based methods have been successfully applied to biomedical entity normalization, they often depend on traditional context-independent word embeddings. Bidirectional Encoder Representations from Transformers (BERT), BERT for Biomedical Text Mining (BioBERT) and BERT for Clinical Text Mining (ClinicalBERT) were recently introduced to pre-train contextualized word representation models using bidirectional Transformers, advancing the state-of-the-art for many natural language processing tasks. In this study, we proposed an entity normalization architecture by fine-tuning the pre-trained BERT / BioBERT / ClinicalBERT models and conducted extensive experiments to evaluate the effectiveness of the pre-trained models for biomedical entity normalization using three different types of datasets. Our experimental results show that the best fine-tuned models consistently outperformed previous methods and advanced the state-of-the-art for biomedical entity normalization, with up to 1.17% increase in accuracy.

研究动机与目标

• 解决生物医学实体规范化中的术语变体问题。
• 探索预训练上下文化表示在规范化任务中的有效性。
• 在多个生物医学数据集上评估 BERT 变体以确立性能提升。

提出的方法

• 在生物医学实体规范化任务上微调预训练的 BERT、BioBERT 和 ClinicalBERT 模型。
• 与先前的规范化方法进行比较以评估性能提升。
• 在三种不同数据集类型上进行广泛实验以评估泛化能力。

实验结果

研究问题

• RQ1 微调后的基于 BERT 的模型是否能够在多样化数据集上超越现有的生物医学实体规范化方法？
• RQ2 哪种预训练的 BERT 变体（BERT、BioBERT、ClinicalBERT）能提供最佳的规范化性能？
• RQ3 在不同数据集上，微调相对于传统方法的准确性提升幅度是多少？

主要发现

• 微调后的 BERT、BioBERT 和 ClinicalBERT 模型持续优于先前的方法。
• 最佳的微调模型在生物医学实体规范化任务上达到最先进的准确性。
• 报道的提升包括在各数据集上相较于先前方法，准确性提高最高可达 1.17%。