[Paper Review] Unified Neural Architecture for Drug, Disease and Clinical Entity Recognition

Sunil Kumar Sahu, Ashish Anand|arXiv (Cornell University)|Aug 11, 2017

Biomedical Text Mining and Ontologies4 citations

TL;DR

This paper proposes a unified neural architecture, CWBLSTM, for biomedical and clinical named entity recognition (NER) using hierarchical bidirectional LSTMs (BLSTM) to capture morphological and contextual features, combined with a first-order CRF layer for sequence labeling. The model outperforms state-of-the-art methods on Drug NER, Disease NER, and Clinical NER tasks without using domain-specific features or external dictionaries, demonstrating the effectiveness of character-level embeddings and CRF-based tag dependency learning.

ABSTRACT

Most existing methods for biomedical entity recognition task rely on explicit feature engineering where many features either are specific to a particular task or depends on output of other existing NLP tools. Neural architectures have been shown across various domains that efforts for explicit feature design can be reduced. In this work we propose an unified framework using bi-directional long short term memory network (BLSTM) for named entity recognition (NER) tasks in biomedical and clinical domains. Three important characteristics of the framework are as follows - (1) model learns contextual as well as morphological features using two different BLSTM in hierarchy, (2) model uses first order linear conditional random field (CRF) in its output layer in cascade of BLSTM to infer label or tag sequence, (3) model does not use any domain specific features or dictionary, i.e., in another words, same set of features are used in the three NER tasks, namely, disease name recognition (Disease NER), drug name recognition (Drug NER) and clinical entity recognition (Clinical NER). We compare performance of the proposed model with existing state-of-the-art models on the standard benchmark datasets of the three tasks. We show empirically that the proposed framework outperforms all existing models. Further our analysis of CRF layer and word-embedding obtained using character based embedding show their importance.

Motivation & Objective

• To develop a unified deep learning framework for biomedical and clinical NER that reduces reliance on task-specific feature engineering.
• To investigate the impact of character-level embeddings and CRF-based sequence modeling on NER performance in low-resource and noisy clinical text.
• To evaluate whether a single model architecture can effectively handle diverse entity types—drugs, diseases, and clinical entities—without domain-specific features or external dictionaries.
• To analyze the complementary roles of morphological (character-level) and semantic (word-level) embeddings in improving NER accuracy.

Proposed method

• The model employs a hierarchical architecture with two BLSTM layers: the first processes character sequences to learn morphologically rich word representations, and the second processes word-level sequences to capture contextual dependencies.
• Word embeddings are learned via a character-level BLSTM followed by a word-level BLSTM, enabling joint modeling of morphology and context.
• A first-order linear-chain CRF is used in the output layer to model label dependencies across the sequence, improving tagging consistency.
• The model uses only word and character embeddings as input, without any manually engineered features, dictionaries, or external tools.
• The framework is trained end-to-end on three standard NER benchmarks: Drug NER, Disease NER, and Clinical NER.
• Model variants are evaluated to ablate the contributions of CRF, bidirectional LSTMs, and character-level embeddings.

Experimental results

Research questions

• RQ1 Can a unified deep learning model achieve state-of-the-art performance across drug, disease, and clinical entity recognition without task-specific features or external knowledge?
• RQ2 How do character-level embeddings contribute to capturing morphological patterns in biomedical entity names compared to standard word embeddings?
• RQ3 To what extent does incorporating CRF-based sequence modeling improve NER performance by modeling label dependencies?
• RQ4 Does the use of bidirectional LSTMs enhance performance compared to unidirectional LSTMs in biomedical NER tasks?
• RQ5 How do the learned character-level and word-level embeddings differ in their semantic and morphological properties?

Key findings

• The proposed CWBLSTM model achieves an F1 score of 83.35 on Disease NER, 73.57 on Drug NER, and 83.68 on Clinical NER, outperforming all existing state-of-the-art models on the respective benchmarks.
• The removal of the CRF layer (BLSTM+WLL variant) results in a 7.5% drop in F1 on Disease NER, 3.4% on Drug NER, and 5.5% on Clinical NER, demonstrating the importance of modeling label dependencies.
• Using bidirectional LSTMs instead of unidirectional LSTMs leads to a 12.89% improvement in F1 on Disease NER, 4.86% on Drug NER, and 20.83% on Clinical NER, highlighting the benefit of bidirectional context.
• Analysis of nearest neighbors shows that character-level embeddings capture morphological similarity (e.g., 'Cefaclor' and 'Cefdinir'), while word-level embeddings preserve semantic similarity (e.g., 'synergistic' and 'antagonistic').
• The model achieves superior performance without using any domain-specific features, dictionaries, or external NLP tools, proving its generalization capability across entity types.
• The average entity length (1.2 for drugs, 2.1 for clinical, 2.2 for diseases) correlates with performance gains from CRF, indicating that longer entities benefit more from sequence modeling.