[Paper Review] Towards Computational Linguistics in Minangkabau Language: Studies on Sentiment Analysis and Machine Translation

Fajri Koto, Ikhwan Koto|arXiv (Cornell University)|Sep 19, 2020

Natural Language Processing Techniques48 references4 citations

TL;DR

This paper presents the first computational linguistics research on the Minangkabau language, introducing a sentiment analysis corpus and a Minangkabau–Indonesian parallel corpus for machine translation. Using classic machine learning and sequence-to-sequence models (LSTM, Transformer), the study finds that word-to-word translation with a bilingual dictionary outperforms neural models in BLEU scores, while cross-lingual transfer from Indonesian to Minangkabau fails significantly, highlighting the need for language-specific resources.

ABSTRACT

Although some linguists (Rusmali et al., 1985; Crouch, 2009) have fairly attempted to define the morphology and syntax of Minangkabau, information processing in this language is still absent due to the scarcity of the annotated resource. In this work, we release two Minangkabau corpora: sentiment analysis and machine translation that are harvested and constructed from Twitter and Wikipedia. We conduct the first computational linguistics in Minangkabau language employing classic machine learning and sequence-to-sequence models such as LSTM and Transformer. Our first experiments show that the classification performance over Minangkabau text significantly drops when tested with the model trained in Indonesian. Whereas, in the machine translation experiment, a simple word-to-word translation using a bilingual dictionary outperforms LSTM and Transformer model in terms of BLEU score.

Motivation & Objective

• To address the lack of computational linguistic resources for Minangkabau, a low-resource Austronesian language with ~7 million speakers.
• To develop and release the first publicly available Minangkabau–Indonesian parallel corpus for machine translation and sentiment analysis.
• To evaluate the performance of classic machine learning and neural sequence-to-sequence models (LSTM, Transformer) on Minangkabau NLP tasks.
• To demonstrate that cross-lingual transfer from Indonesian to Minangkabau fails significantly, necessitating language-specific training data.
• To preserve and promote Minangkabau language through NLP by creating benchmark datasets and models.

Proposed method

• Constructed a Minangkabau–Indonesian bilingual dictionary by manually translating the top 20,000 most frequent words from Minangkabau Wikipedia into Indonesian.
• Built a sentiment analysis corpus by manually translating 5,000 Indonesian sentences into Minangkabau, creating a parallel benchmark for classification.
• Developed benchmark models using traditional machine learning and pre-trained language models for Minangkabau sentiment classification.
• Automatically created a high-quality parallel corpus of 16,000 Minangkabau–Indonesian sentence pairs from Wikipedia and Twitter.
• Trained and evaluated LSTM and Transformer sequence-to-sequence models for machine translation with shared word embeddings and cosine positional encoding.
• Evaluated models using BLEU scores and conducted human evaluation on fluency and adequacy to compare word-to-word translation, LSTM, and Transformer outputs.

Experimental results

Research questions

• RQ1 Can existing Indonesian NLP models effectively classify Minangkabau text, or is a language-specific model required?
• RQ2 Does neural sequence-to-sequence translation (LSTM, Transformer) outperform a simple word-to-word translation using a bilingual dictionary in Minangkabau–Indonesian translation?
• RQ3 Why do sequence-to-sequence models underperform on Minangkabau–Indonesian translation, especially on out-of-domain data?
• RQ4 To what extent does vocabulary overlap between Minangkabau and Indonesian affect translation performance?
• RQ5 How do fluency and adequacy of translations compare between rule-based (word-to-word) and neural models in low-resource settings?

Key findings

• The word-to-word (W2W) translation using a bilingual dictionary achieved the highest BLEU scores, with 64.54 on the Wikipedia test set for Minangkabau-to-Indonesian translation.
• LSTM and Transformer models significantly underperformed compared to W2W, with BLEU scores dropping to 63.77 and 56.25 respectively on the same test set.
• In the sentiment analysis task, models trained on Indonesian data failed to generalize to Minangkabau, indicating a lack of cross-lingual transferability between the two languages.
• The performance drop in sequence-to-sequence models was primarily due to out-of-vocabulary (OOV) issues, with ~65% of words in the sentiment corpus not present in the training vocabulary.
• Human evaluation confirmed that W2W translation was significantly more adequate than LSTM, though fluency was similar, suggesting that LSTM models hallucinated incorrect keywords despite fluent output.
• The study demonstrates that for low-resource languages like Minangkabau, rule-based translation with curated dictionaries can outperform complex neural models without large-scale parallel data.