[Paper Review] CLUENER2020: Fine-grained Named Entity Recognition Dataset and Benchmark for Chinese

Liang Xu, Tong Yu+8 more|arXiv (Cornell University)|Jan 13, 2020

Topic Modeling12 references48 citations

TL;DR

The paper introduces CLUENER2020, a fine-grained Chinese NER dataset with 10 entity categories, and provides baselines and human performance analysis to facilitate future research.

ABSTRACT

In this paper, we introduce the NER dataset from CLUE organization (CLUENER2020), a well-defined fine-grained dataset for named entity recognition in Chinese. CLUENER2020 contains 10 categories. Apart from common labels like person, organization, and location, it contains more diverse categories. It is more challenging than current other Chinese NER datasets and could better reflect real-world applications. For comparison, we implement several state-of-the-art baselines as sequence labeling tasks and report human performance, as well as its analysis. To facilitate future work on fine-grained NER for Chinese, we release our dataset, baselines, and leader-board.

Motivation & Objective

• Define a fine-grained Chinese NER dataset with diverse entity categories.
• Provide a benchmark and baseline models for fair comparison on Chinese NER.
• Analyze the difficulty and real-world applicability of fine-grained NER in Chinese.

Proposed method

• Assemble and annotate a Chinese NER dataset with 10 entity categories beyond common labels.
• Implement several state-of-the-art baseline models as sequence labeling tasks.
• Evaluate baselines and human performance on the CLUENER2020 dataset.
• Release the dataset, baselines, and leaderboard to the community.

Experimental results

Research questions

• RQ1 What are the benefits and challenges of fine-grained NER in Chinese compared to coarser-grained labels?
• RQ2 How do modern sequence labeling baselines perform on the CLUENER2020 dataset across multiple entity categories?
• RQ3 How does human performance compare to automated baselines on this fine-grained Chinese NER task?

Key findings

• CLUENER2020 provides a well-defined fine-grained Chinese NER dataset covering ten categories.
• Baseline models are evaluated on the dataset to establish a competitive benchmark.
• Human performance is reported to contextualize model performance against expert labeling.
• The dataset, baselines, and leaderboard are released to facilitate future work.