[Paper Review] A Survey on Deep Learning for Named Entity Recognition

Jing Li, Aixin Sun+2 more|arXiv (Cornell University)|Dec 22, 2018

Topic Modeling194 references100 citations

TL;DR

The paper surveys deep learning techniques for NER, organizing approaches by input representations, context encoders, and tag decoders, and discusses datasets, tools, evaluation, challenges, and future directions.

ABSTRACT

Named entity recognition (NER) is the task to identify mentions of rigid designators from text belonging to predefined semantic types such as person, location, organization etc. NER always serves as the foundation for many natural language applications such as question answering, text summarization, and machine translation. Early NER systems got a huge success in achieving good performance with the cost of human engineering in designing domain-specific features and rules. In recent years, deep learning, empowered by continuous real-valued vector representations and semantic composition through nonlinear processing, has been employed in NER systems, yielding stat-of-the-art performance. In this paper, we provide a comprehensive review on existing deep learning techniques for NER. We first introduce NER resources, including tagged NER corpora and off-the-shelf NER tools. Then, we systematically categorize existing works based on a taxonomy along three axes: distributed representations for input, context encoder, and tag decoder. Next, we survey the most representative methods for recent applied techniques of deep learning in new NER problem settings and applications. Finally, we present readers with the challenges faced by NER systems and outline future directions in this area.

Motivation & Objective

• Motivate and define NER and its significance for NLP applications.
• Summarize available English NER datasets and off-the-shelf tools.
• Propose a taxonomy of DL-based NER by input representations, context encoders, and tag decoders.
• Survey representative deep learning techniques and their applicability to varied NER settings.
• Discuss challenges and outline future research directions in DL-based NER.

Proposed method

• Introduce a three-axis taxonomy for DL-based NER: distributed input representations, context encoders (CNN/RNN), and tag decoders.
• Review word-level, character-level, and hybrid representations used in NER models.
• Discuss context encoder architectures (e.g., CNNs, RNNs, LSTMs, BiLSTMs) and how they interact with tag decoding.
• Summarize dataset resources (e.g., CoNLL03, OntoNotes) and widely-used NER tools (academia and industry).
• Explain evaluation metrics (exact-match and relaxed-match) and traditional evaluation schemes and their trade-offs.

Experimental results

Research questions

• RQ1 What are the prevalent deep learning architectures and representations used for NER, and how do they impact performance?
• RQ2 Which resources (datasets and tools) are most influential for English NER research and benchmarking?
• RQ3 How should we evaluate NER systems, and what are the strengths and weaknesses of different evaluation schemes?
• RQ4 What are the main challenges and open directions for DL-based NER in practice and research?

Key findings

• Deep learning has become dominant in NER due to automatic representation learning and end-to-end training.
• Word-level, character-level, and hybrid (including gazetteers and contextualized embeddings) representations are central to DL-based NER.
• Context encoders (CNNs, RNNs, LSTMs, transformers) and tag decoders jointly determine NER performance, with context modeling being crucial.
• A variety of English NER datasets and off-the-shelf tools are available, spanning news, web, biomedical, and user-generated text.
• Evaluation frameworks include exact-match (boundary and type) and relaxed-match schemes, with trade-offs in interpretability and comparability.