[Paper Review] CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition

Yuying Zhu, Guoxin Wang+1 more|arXiv (Cornell University)|Apr 3, 2019

Topic Modeling43 references68 citations

TL;DR

CAN-NER introduces a character-based CNN with a local convolutional attention layer plus a BiGRU-CRF with a global self-attention layer to perform Chinese NER without word embeddings or external lexicons, achieving state-of-the-art results across multiple domains.

ABSTRACT

Named entity recognition (NER) in Chinese is essential but difficult because of the lack of natural delimiters. Therefore, Chinese Word Segmentation (CWS) is usually considered as the first step for Chinese NER. However, models based on word-level embeddings and lexicon features often suffer from segmentation errors and out-of-vocabulary (OOV) words. In this paper, we investigate a Convolutional Attention Network called CAN for Chinese NER, which consists of a character-based convolutional neural network (CNN) with local-attention layer and a gated recurrent unit (GRU) with global self-attention layer to capture the information from adjacent characters and sentence contexts. Also, compared to other models, not depending on any external resources like lexicons and employing small size of char embeddings make our model more practical. Extensive experimental results show that our approach outperforms state-of-the-art methods without word embedding and external lexicon resources on different domain datasets including Weibo, MSRA and Chinese Resume NER dataset.

Motivation & Objective

• Motivate robust Chinese NER without reliance on word segmentation, embeddings, or lexicons.
• Develop a character-level model that captures local context and long-range dependencies.
• Integrate a convolutional attention mechanism to enhance local character relationships.
• Incorporate a global self-attention layer to model sentence-level context.
• Demonstrate effectiveness across diverse domains without external resources.

Proposed method

• Use a BiGRU-CRF backbone as the main sequence labeling framework.
• Add a convolutional attention layer to encode local contextual information around each character within a window.
• Concatenate segmentation information (BMES) with character embeddings as input to the CNN.
• Compute local attention weights within each window to form hidden representations.
• Employ a global self-attention layer over BiGRU outputs to capture long-range dependencies.
• Finish with a CRF layer on top of concatenated BiGRU and global-attention outputs for decoding.

Experimental results

Research questions

• RQ1 Can a fully character-based model achieve competitive NER performance without word embeddings or lexicons across multiple domains?
• RQ2 Does a local convolutional attention mechanism improve modeling of nearby character interactions compared to standard CNNs?
• RQ3 Does a global self-attention layer effectively capture long-range sentence dependencies to boost NER performance in Chinese?
• RQ4 How does the proposed CAN-NER perform relative to state-of-the-art models on Weibo, MSRA, Chinese Resume, and OntoNotes datasets without external resources?

Key findings

• CAN-NER outperforms baselines and achieves state-of-the-art results among character-based models across several datasets.
• Convolutional Attention provides notable improvements over standard CNN features by better capturing local character relationships.
• Global self-attention on BiGRU outputs helps model long-range sentence context beyond the capabilities of vanilla BiGRU-CRF, improving F1 scores.
• The model operates without external word embeddings or lexicon resources, offering a more practical NER solution.
• Results show strong performance on Weibo and Chinese Resume datasets, with competitive outcomes on MSRA and OntoNotes datasets.