[Paper Review] A Survey on Recent Advances in Named Entity Recognition from Deep Learning models

Vikas Yadav, Steven Bethard|arXiv (Cornell University)|Oct 25, 2019

Topic Modeling69 references468 citations

TL;DR

This survey analyzes deep neural network architectures for NER, compares them to feature-engineered approaches, and highlights the progress and key insights up to neural models for multilingual and multi-domain NER.

ABSTRACT

Named Entity Recognition (NER) is a key component in NLP systems for question answering, information retrieval, relation extraction, etc. NER systems have been studied and developed widely for decades, but accurate systems using deep neural networks (NN) have only been introduced in the last few years. We present a comprehensive survey of deep neural network architectures for NER, and contrast them with previous approaches to NER based on feature engineering and other supervised or semi-supervised learning algorithms. Our results highlight the improvements achieved by neural networks, and show how incorporating some of the lessons learned from past work on feature-based NER systems can yield further improvements.

Motivation & Objective

• Synthesize the evolution from feature-based to neural NER systems and quantify performance gains from neural architectures.
• Contrast neural approaches with knowledge-based and feature-engineered methods across multiple languages and domains.
• Identify architectural patterns and lessons that improve NER performance.
• Provide guidance and resources for replicating and extending neural NER systems.

Proposed method

• Review 154 articles and select 83 to cover neural architectures and feature-engineered baselines.
• Categorize NER systems into knowledge-based, bootstrapped, feature-engineered, and neural network approaches.
• Summarize datasets, evaluation metrics, and shared tasks to contextualize performance.
• Compare word-level, character-level, and hybrid representations for NER.
• Highlight the role of lexicons, gazetteers, and affixes in neural NER.
• Report cross-language and cross-domain performance trends from CoNLL, DrugNER, and biomedical datasets.

Experimental results

Research questions

• RQ1 What performance gains do neural network NER systems achieve over traditional feature-engineered models across languages and domains?
• RQ2 How do word-level, character-level, and hybrid representations compare for NER accuracy?
• RQ3 What is the impact of incorporating lexicons, gazetteers, and affix features into neural NER architectures?
• RQ4 How do neural NER models perform on multilingual (CoNLL) and biomedical/drug-related datasets (DrugNER)?
• RQ5 What lessons from past feature-based NER can further improve modern neural approaches?

Key findings

• Neural-network NER systems generally outperform feature-engineered models on the evaluated datasets.
• Word+character hybrid models often outperform purely word-based or purely character-based models.
• In DrugNER, word+character models achieve notably higher F1 scores than feature-engineered baselines.
• An affix-enhanced character+word model sets new state-of-the-art results for Spanish, Dutch, and German, and closely approaches English performance.
• Incorporating past feature-engineering insights (like affixes) can yield meaningful gains in neural NER systems.