[Paper Review] Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of Marathi

Saurabh Gaikwad, Tharindu Ranasinghe+2 more|Aston Publications Explorer (Aston University)|Sep 8, 2021

Hate Speech and Cyberbullying Detection40 references4 citations

TL;DR

This paper introduces MOLD, the first offensive language identification dataset for Marathi, a low-resource Indo-Aryan language. Using cross-lingual transfer learning with XLM-RoBERTa, it demonstrates that fine-tuning on similar languages like Hindi and Bengali significantly improves performance over English-based transfer, with Hindi achieving a macro F1 of 0.9401 in transfer learning and 0.8396 in zero-shot settings.

ABSTRACT

The widespread presence of offensive language on social media motivated the development of systems capable of recognizing such content automatically. Apart from a few notable exceptions, most research on automatic offensive language identification has dealt with English. To address this shortcoming, we introduce MOLD, the Marathi Offensive Language Dataset. MOLD is the first dataset of its kind compiled for Marathi, thus opening a new domain for research in low-resource Indo-Aryan languages. We present results from several machine learning experiments on this dataset, including zero-short and other transfer learning experiments on state-of-the-art cross-lingual transformers from existing data in Bengali, English, and Hindi.

Motivation & Objective

• Address the lack of offensive language identification resources for low-resource Indo-Aryan languages, particularly Marathi.
• Develop and release MOLD, the first annotated dataset of ~2,500 Marathi tweets for offensive language identification.
• Investigate the effectiveness of cross-lingual transfer learning using multilingual transformers (XLM-R) for low-resource language tasks.
• Examine the role of language similarity in cross-lingual transfer performance, comparing English, Hindi, and Bengali as source languages.
• Provide open-source models and code to support reproducibility and further research in low-resource offensive language detection.

Proposed method

• Collected and manually annotated 2,500 Marathi tweets for offensive language, creating the MOLD dataset.
• Trained monolingual models (SVM, LSTM) and fine-tuned XLM-RoBERTa on the MOLD dataset for offensive language classification.
• Applied transfer learning by initializing XLM-R models with pre-trained weights from English (OLID, SOLID), Hindi (HASOC), and Bengali (TRAC) datasets.
• Performed zero-shot inference by testing models trained on other languages directly on the Marathi test set without fine-tuning.
• Conducted few-shot learning experiments by fine-tuning models on increasing subsets (100–4,000 instances) of the Marathi dataset.
• Evaluated performance using macro and weighted F1 scores across multiple transfer learning and zero-shot settings.

Experimental results

Research questions

• RQ1 RQ1: What is the impact of dataset size on monolingual and cross-lingual models for offensive language identification in low-resource settings?
• RQ2 RQ2: How does language similarity influence cross-lingual prediction performance in offensive language identification?
• RQ3 RQ3: How do transfer learning strategies from English, Hindi, and Bengali compare in predicting offensive content in Marathi?
• RQ4 RQ4: Does increasing the size of the source training data beyond a certain point yield diminishing returns in transfer learning performance?

Key findings

• Transfer learning from Hindi achieved the highest macro F1 score of 0.9401 on the Marathi test set, outperforming all other source languages and monolingual baselines.
• Hindi-based transfer learning also achieved the best zero-shot performance with a macro F1 of 0.8396, indicating strong language similarity benefits.
• Despite being over eight times larger, the EN-SOLID dataset yielded lower performance (macro F1 = 0.9321) than the smaller Hindi dataset (0.9401), suggesting that language similarity outweighs dataset size.
• Few-shot learning experiments confirmed that Hindi consistently outperformed other languages across all shot settings, reinforcing the advantage of linguistic proximity.
• The performance gap between transfer learning from OLID and SOLID was minimal, indicating that beyond a certain data threshold, additional data does not significantly improve results.
• The results demonstrate that cross-lingual transfer from closely related languages like Hindi and Marathi is more effective than from distant languages like English, even when the latter have larger training sets.