[Paper Review] An Open-Source Dataset and A Multi-Task Model for Malay Named Entity Recognition

Yingwen Fu, Nankai Lin+2 more|arXiv (Cornell University)|Sep 3, 2021

Topic Modeling24 references5 citations

TL;DR

This paper introduces MYNER, a large-scale open-source Malay Named Entity Recognition (NER) dataset with 28,991 sentences and 384k+ tokens, constructed via a framework leveraging homologous language data and iterative optimization. It further proposes a multi-task learning model with a bidirectional revision mechanism and gated ignoring for boundary detection, achieving state-of-the-art performance on MYNER and establishing a benchmark for low-resource Malay NLP.

ABSTRACT

Named entity recognition (NER) is a fundamental task of natural language processing (NLP). However, most state-of-the-art research is mainly oriented to high-resource languages such as English and has not been widely applied to low-resource languages. In Malay language, relevant NER resources are limited. In this work, we propose a dataset construction framework, which is based on labeled datasets of homologous languages and iterative optimization, to build a Malay NER dataset (MYNER) comprising 28,991 sentences (over 384 thousand tokens). Additionally, to better integrate boundary information for NER, we propose a multi-task (MT) model with a bidirectional revision (Bi-revision) mechanism for Malay NER task. Specifically, an auxiliary task, boundary detection, is introduced to improve NER training in both explicit and implicit ways. Furthermore, a gated ignoring mechanism is proposed to conduct conditional label transfer and alleviate error propagation by the auxiliary task. Experimental results demonstrate that our model achieves comparable results over baselines on MYNER. The dataset and the model in this paper would be publicly released as a benchmark dataset.

Motivation & Objective

• To address the scarcity of high-quality, large-scale NER resources for low-resource languages like Malay.
• To develop a scalable, data-efficient framework for constructing high-quality NER datasets in low-resource settings.
• To improve Malay NER performance by integrating boundary detection as an auxiliary task in a multi-task learning setup.
• To mitigate error propagation from auxiliary tasks through a conditional label transfer mechanism.
• To establish a publicly available benchmark dataset and model for future research in Malay NLP.

Proposed method

• A dataset construction framework that leverages labeled data from related high-resource languages and applies iterative optimization to refine and expand the Malay NER dataset.
• The creation of MYNER, a final dataset of 28,991 sentences and over 384,000 tokens, annotated with six entity types.
• Design of a multi-task model with a bidirectional revision (Bi-revision) mechanism to jointly optimize NER and boundary detection.
• Incorporation of a gated ignoring mechanism that selectively transfers labels from the boundary detection task to reduce error propagation.
• Training the model end-to-end with shared encoder and task-specific heads, where the boundary detection task enhances NER via explicit and implicit supervision.
• Use of bidirectional LSTM and CRF layers to model sequential dependencies and improve tagging accuracy.

Experimental results

Research questions

• RQ1 Can a data construction framework based on homologous language resources and iterative refinement produce a high-quality, large-scale NER dataset for low-resource languages like Malay?
• RQ2 Does introducing boundary detection as an auxiliary task improve the performance of Malay NER models?
• RQ3 Can a gated ignoring mechanism effectively reduce error propagation from the auxiliary boundary detection task during multi-task training?
• RQ4 How does the proposed multi-task model with Bi-revision compare to strong baselines on the newly constructed MYNER dataset?
• RQ5 To what extent does the proposed model and dataset serve as a viable benchmark for future Malay NLP research?

Key findings

• The proposed dataset construction framework successfully generated MYNER, a high-quality, large-scale Malay NER dataset with 28,991 sentences and over 384,000 tokens.
• The multi-task model with bidirectional revision and gated ignoring outperformed strong baseline models on the MYNER dataset in terms of F1 score.
• The auxiliary boundary detection task contributed to improved NER performance through both explicit and implicit supervision mechanisms.
• The gated ignoring mechanism effectively reduced error propagation from the auxiliary task, enhancing overall training stability and performance.
• The model achieved state-of-the-art results on the MYNER benchmark, validating its effectiveness for low-resource Malay NLP.
• The dataset and model are publicly released, establishing a new benchmark for future research in Malay NER.