[Paper Review] Robust Sentiment Analysis for Low Resource languages Using Data Augmentation Approaches: A Case Study in Marathi

Aabha Pingle, Aditya Vyawahare+4 more|arXiv (Cornell University)|Oct 1, 2023

Sentiment Analysis and Opinion MiningComputer Science3 citations

TL;DR

This paper proposes four data augmentation techniques—paraphrasing, back-translation, BERT-based token replacement (including named entity and random masking), and GPT-based text/label generation—to improve sentiment analysis in low-resource Marathi. It achieves significant performance gains, with the GPT-based label generation approach reaching 84.35% accuracy on the in-domain MahaSent dataset, demonstrating improved robustness and generalization across domains.

ABSTRACT

Sentiment analysis plays a crucial role in understanding the sentiment expressed in text data. While sentiment analysis research has been extensively conducted in English and other Western languages, there exists a significant gap in research efforts for sentiment analysis in low-resource languages. Limited resources, including datasets and NLP research, hinder the progress in this area. In this work, we present an exhaustive study of data augmentation approaches for the low-resource Indic language Marathi. Although domain-specific datasets for sentiment analysis in Marathi exist, they often fall short when applied to generalized and variable-length inputs. To address this challenge, this research paper proposes four data augmentation techniques for sentiment analysis in Marathi. The paper focuses on augmenting existing datasets to compensate for the lack of sufficient resources. The primary objective is to enhance sentiment analysis model performance in both in-domain and cross-domain scenarios by leveraging data augmentation strategies. The data augmentation approaches proposed showed a significant performance improvement for cross-domain accuracies. The augmentation methods include paraphrasing, back-translation; BERT-based random token replacement, named entity replacement, and pseudo-label generation; GPT-based text and label generation. Furthermore, these techniques can be extended to other low-resource languages and for general text classification tasks.

Motivation & Objective

• Address the scarcity of labeled datasets for sentiment analysis in low-resource Indic languages, particularly Marathi.
• Improve model generalization and robustness in sentiment classification by augmenting existing Marathi datasets with synthetic data.
• Evaluate the effectiveness of data augmentation techniques in both in-domain and cross-domain sentiment analysis scenarios.
• Demonstrate that data augmentation can significantly enhance model performance despite limited annotated data in Marathi.
• Provide a transferable framework applicable to other low-resource languages and text classification tasks.

Proposed method

• Employed BERT-based random token replacement and named entity replacement to generate syntactically varied but sentiment-preserving sentences.
• Applied back-translation using English as an intermediate language to generate diverse paraphrases while preserving sentiment.
• Utilized GPT-based models to generate both synthetic text and corresponding sentiment labels, enhancing data diversity and label consistency.
• Implemented sequential masking strategies using BERT to replace tokens with masked or masked-replaced tokens, preserving semantic structure.
• Fine-tuned BERT and GPT models on augmented datasets to evaluate performance across in-domain and cross-domain settings.
• Used confusion matrices and standard metrics (accuracy, F1-score) to compare model performance across augmentation techniques.

Experimental results

Research questions

• RQ1 Can data augmentation techniques significantly improve sentiment classification accuracy in low-resource Marathi NLP tasks?
• RQ2 How do different data augmentation strategies (e.g., back-translation, BERT masking, GPT generation) compare in enhancing model generalization across domains?
• RQ3 To what extent does knowledge transfer between domains (e.g., training on GoEmotions, testing on MahaSent) improve performance in Marathi sentiment analysis?
• RQ4 Which augmentation method yields the highest in-domain accuracy on the MahaSent dataset?
• RQ5 Can synthetic data generation preserve sentiment polarity while increasing dataset diversity and model robustness?

Key findings

• The GPT-based label generation approach achieved the highest in-domain accuracy of 84.35% on the MahaSent test set, outperforming all other methods.
• The BERT-based random masking strategy improved accuracy from 83.67% (baseline) to 84.30% on the MahaSent dataset, demonstrating effectiveness in data augmentation.
• Cross-domain performance improved significantly, with models fine-tuned on GoEmotions achieving 73.62% accuracy on the MahaSent test set, indicating successful knowledge transfer.
• The BERT-based named entity replacement method showed notable gains in cross-domain generalization, enhancing model robustness across varying input distributions.
• The GPT-based approach 1 (label generation) achieved 63.20% accuracy on the GoEmotions test set, indicating strong performance in the target domain.
• Confusion matrices revealed consistent improvements in class-wise precision and recall across all augmentation techniques, especially for imbalanced sentiment labels.