[Paper Review] Improving English to Sinhala Neural Machine Translation using Part-of-Speech Tag

Ravinga Perera, Thilakshi Fonseka+2 more|arXiv (Cornell University)|Feb 17, 2022

Natural Language Processing Techniques4 citations

TL;DR

This paper proposes enhancing English-to-Sinhala neural machine translation (NMT) by integrating Part-of-Speech (POS) tags into the Transformer model's input embeddings and positional encoding. The best-performing model, which concatenates POS embeddings with subword embeddings, achieves a BLEU score of 30.84—0.92 higher than the baseline—demonstrating that linguistic features improve translation quality in low-resource settings.

ABSTRACT

The performance of Neural Machine Translation (NMT) depends significantly on the size of the available parallel corpus. Due to this fact, low resource language pairs demonstrate low translation performance compared to high resource language pairs. The translation quality further degrades when NMT is performed for morphologically rich languages. Even though the web contains a large amount of information, most people in Sri Lanka are unable to read and understand English properly. Therefore, there is a huge requirement of translating English content to local languages to share information among locals. Sinhala language is the primary language in Sri Lanka and building an NMT system that can produce quality English to Sinhala translations is difficult due to the syntactic divergence between these two languages under low resource constraints. Thus, in this research, we explore effective methods of incorporating Part of Speech (POS) tags to the Transformer input embedding and positional encoding to further enhance the performance of the baseline English to Sinhala neural machine translation model.

Motivation & Objective

• To address the low translation performance of English-to-Sinhala neural machine translation due to syntactic divergence and limited parallel corpora.
• To investigate whether incorporating Part-of-Speech (POS) tagging improves translation quality in low-resource language pairs.
• To evaluate two distinct integration methods: POS embedding in input representation and POS in positional encoding.
• To develop a domain-specific NMT system using government documents as the parallel corpus.
• To establish a benchmark for POS-integrated NMT in low-resource, morphologically rich languages like Sinhala.

Proposed method

• The Transformer architecture is used as the base NMT model, with input embeddings extended by concatenating subword embeddings and POS embeddings.
• POS tags are assigned to all training, validation, and test sets using the Stanford POS tagger for English.
• Two integration strategies are evaluated: (1) combining POS embeddings with input embeddings, and (2) modifying positional encoding with POS information.
• Byte-Pair Encoding (BPE) is applied to all datasets to handle OOV and rare words.
• Hyperparameters are kept consistent with the baseline model to isolate the effect of POS integration.
• The model is trained using Adam optimizer with standard learning rates and beam search decoding (beam size=5, length penalty=1.2).

Experimental results

Research questions

• RQ1 Can integrating Part-of-Speech (POS) tags into the input embeddings of a Transformer NMT model improve translation quality for the low-resource English-to-Sinhala language pair?
• RQ2 Does incorporating POS information into positional encoding enhance or degrade NMT performance in morphologically rich, low-resource languages?
• RQ3 How does POS integration compare to existing linguistic feature integration in low-resource NMT systems in terms of BLEU score improvement?
• RQ4 Is the performance gain from POS integration robust across different integration strategies within the Transformer framework?
• RQ5 Can a domain-specific parallel corpus from government documents support effective POS-augmented NMT training for Sinhala?

Key findings

• The model that integrates POS tags into input embeddings achieves a BLEU score of 30.84, representing a 0.92 improvement over the baseline model's score of 29.92.
• The model integrating POS into positional encoding underperforms the baseline, achieving a BLEU score of 28.75, indicating that such modification disrupts the original positional information.
• The improvement from POS integration is significant in the context of low-resource NMT, where gains are typically marginal.
• The best-performing model produces more accurate and contextually appropriate Sinhala translations, as shown in qualitative examples from the test set.
• POS tagging helps disambiguate word senses (e.g., 'book' as noun or verb), leading to better morphological alignment in the target language.
• The results confirm that linguistic features such as POS tags can effectively enhance NMT performance even with limited parallel data.