[Paper Review] COVID-19-related Nepali Tweets Classification in a Low Resource Setting

Rabin Adhikari, Safal Thapaliya+4 more|arXiv (Cornell University)|Oct 11, 2022

Misinformation and Its Impacts4 citations

TL;DR

This paper proposes a low-resource NLP pipeline for classifying Nepali tweets related to COVID-19 into eight key topics using multilingual transformer models (mBERT and MuRIL). It implements a web-based dashboard to visualize trends, demonstrating that MuRIL outperforms mBERT on larger datasets, with all data, models, and tools open-sourced for public use.

ABSTRACT

Billions of people across the globe have been using social media platforms in their local languages to voice their opinions about the various topics related to the COVID-19 pandemic. Several organizations, including the World Health Organization, have developed automated social media analysis tools that classify COVID-19-related tweets into various topics. However, these tools that help combat the pandemic are limited to very few languages, making several countries unable to take their benefit. While multi-lingual or low-resource language-specific tools are being developed, they still need to expand their coverage, such as for the Nepali language. In this paper, we identify the eight most common COVID-19 discussion topics among the Twitter community using the Nepali language, set up an online platform to automatically gather Nepali tweets containing the COVID-19-related keywords, classify the tweets into the eight topics, and visualize the results across the period in a web-based dashboard. We compare the performance of two state-of-the-art multi-lingual language models for Nepali tweet classification, one generic (mBERT) and the other Nepali language family-specific model (MuRIL). Our results show that the models' relative performance depends on the data size, with MuRIL doing better for a larger dataset. The annotated data, models, and the web-based dashboard are open-sourced at https://github.com/naamiinepal/covid-tweet-classification.

Motivation & Objective

• To identify the eight most common COVID-19 discussion topics in Nepali tweets.
• To develop an automated pipeline for collecting, classifying, and visualizing Nepali social media content during the pandemic.
• To evaluate the performance of multilingual and Nepali-optimized language models (mBERT and MuRIL) on low-resource Nepali text.
• To create and release a publicly accessible web-based dashboard for real-time monitoring of public sentiment and topic trends.

Proposed method

• Constructed a keyword-based web crawler to collect Nepali tweets containing COVID-19-related terms from Twitter.
• Annotated a dataset of Nepali tweets into eight predefined topics based on thematic content.
• Fine-tuned two multilingual transformer models—mBERT and MuRIL—for text classification using the annotated dataset.
• Implemented a web-based dashboard to visualize topic distribution and temporal trends of classified tweets.
• Compared model performance using standard classification metrics, analyzing the impact of dataset size on model accuracy.
• Open-sourced the annotated dataset, trained models, and dashboard code for reuse and extension by the research community.

Experimental results

Research questions

• RQ1 What are the eight most prevalent topics of discussion in Nepali tweets related to COVID-19?
• RQ2 How do multilingual models like mBERT and language-specific models like MuRIL perform on low-resource Nepali text classification?
• RQ3 Does dataset size significantly influence the relative performance of mBERT versus MuRIL in classifying Nepali tweets?
• RQ4 Can an automated pipeline effectively collect, classify, and visualize real-time public sentiment on social media in low-resource languages?
• RQ5 To what extent can open-sourced tools support public health monitoring in low-resource linguistic settings?

Key findings

• The eight most common topics in Nepali COVID-19 tweets were identified as: symptoms, vaccines, misinformation, government policies, mental health, economic impact, testing, and travel restrictions.
• MuRIL outperformed mBERT on the Nepali tweet classification task when fine-tuned on a larger annotated dataset.
• The performance gap between mBERT and MuRIL narrowed with smaller datasets, indicating that MuRIL’s advantage is more pronounced with sufficient training data.
• The web-based dashboard successfully visualized temporal trends in public discourse, enabling real-time monitoring of topic prevalence.
• The entire pipeline—including data, models, and dashboard—was released as open-source, supporting reproducibility and future extension in low-resource NLP.