[Paper Review] Domain-specific MT for Low-resource Languages: The case of Bambara-French

Allahsera Auguste Tapo, Michael Leventhal+3 more|arXiv (Cornell University)|Mar 31, 2021

Natural Language Processing Techniques15 references4 citations

TL;DR

This paper presents the first domain-specific parallel dataset for Bambara-French machine translation, focusing on medical texts, and evaluates neural machine translation models trained on this data. Despite limited data, combining domain-specific medical texts with general-purpose dictionary data yields better performance than using either dataset alone, suggesting that data diversity enhances low-resource translation models.

ABSTRACT

Translating to and from low-resource languages is a challenge for machine translation (MT) systems due to a lack of parallel data. In this paper we address the issue of domain-specific MT for Bambara, an under-resourced Mande language spoken in Mali. We present the first domain-specific parallel dataset for MT of Bambara into and from French. We discuss challenges in working with small quantities of domain-specific data for a low-resource language and we present the results of machine learning experiments on this data.

Motivation & Objective

• To address the challenge of low-resource machine translation for Bambara, a Mande language spoken in Mali with minimal parallel training data.
• To create and release the first domain-specific parallel dataset for Bambara-French translation, focusing on rural healthcare content.
• To evaluate neural machine translation models trained on this new dataset and compare performance against models trained on general-purpose dictionary data.
• To investigate the impact of data diversity—specifically combining general and domain-specific data—on model performance in low-resource settings.
• To establish benchmarks and best practices for training NMT systems with small, domain-specific datasets for under-resourced languages.

Proposed method

• Constructed a tri-lingual (Bambara, French, English) medical health guide dataset, using content from the Dokotoro Project with permission.
• Performed paragraph-level alignment of Bambara and French texts, with sentence-level alignment validated by expert translators to ensure semantic equivalence.
• Applied byte pair encoding (BPE) with 1000 subword units and BPE dropout to handle out-of-vocabulary words due to Bambara's lack of standard tokenization.
• Trained Transformer-based NMT models with six layers, 1024 model size, 256 hidden size, and 4 attention heads per layer using the Joey NMT framework.
• Used ADAM optimizer with a constant learning rate of 0.0004, dropout of 0.1, and label smoothing of 0.2 for regularization.
• Evaluated models using SacreBLEU and ChrF scores on held-out test sets, with beam search (width 5) during inference.

Experimental results

Research questions

• RQ1 How does training a neural machine translation model on domain-specific medical text compare to training on general-purpose dictionary data for Bambara-French translation?
• RQ2 What is the impact of combining general and domain-specific data on model performance in a low-resource setting?
• RQ3 Can data from different linguistic styles (simple vs. complex medical language) be effectively combined to improve translation quality?
• RQ4 To what extent do standard metrics like BLEU and ChrF reflect translation quality for complex, domain-specific content in low-resource languages?
• RQ5 How do alignment quality and data curation affect the final performance of low-resource NMT systems?

Key findings

• The BLEU and ChrF scores for the medical domain dataset were significantly lower than those for the dictionary dataset, indicating greater difficulty in translating complex medical content.
• A model trained on both dictionary and medical data achieved performance comparable to the model trained only on the dictionary data, suggesting that the richer medical data did not degrade performance.
• Combining general and domain-specific data yielded better results than using either dataset in isolation, indicating that data diversity strengthens low-resource models.
• Models trained on the medical dataset alone performed poorly when evaluated on the dictionary test set, and vice versa, highlighting domain-specificity in performance.
• The results suggest that BLEU and ChrF may be limited metrics for assessing translation quality in complex, domain-specific contexts, especially with small datasets.