[Paper Review] Low resource language dataset creation, curation and classification: Setswana and Sepedi -- Extended Abstract

Vukosi Marivate, Tshephisho Joseph Sefara+5 more|arXiv (Cornell University)|Mar 30, 2020

Natural Language Processing Techniques4 citations

TL;DR

This paper proposes a framework for creating, curating, and classifying low-resource language datasets using Setswana and Sepedi news headlines, employing data augmentation and pre-trained vectorizers to improve classification performance. It achieves notable results with logistic regression and XGBoost on small datasets, demonstrating feasibility for low-resource NLP tasks.

ABSTRACT

The recent advances in Natural Language Processing have only been a boon for well represented languages, negating research in lesser known global languages. This is in part due to the availability of curated data and research resources. One of the current challenges concerning low-resourced languages are clear guidelines on the collection, curation and preparation of datasets for different use-cases. In this work, we take on the task of creating two datasets that are focused on news headlines (i.e short text) for Setswana and Sepedi and the creation of a news topic classification task from these datasets. In this study, we document our work, propose baselines for classification, and investigate an approach on data augmentation better suited to low-resourced languages in order to improve the performance of the classifiers.

Motivation & Objective

• Address the lack of curated linguistic resources for low-resource languages, particularly in Southern Africa.
• Create annotated news headline datasets for Setswana and Sepedi, two underrepresented South African languages.
• Develop a classification framework for categorizing short news headlines into predefined topics such as politics, sports, and crime.
• Investigate data augmentation techniques suitable for low-resource languages to improve classifier generalization.
• Establish baselines for future research on Setswana and Sepedi NLP tasks.

Proposed method

• Collected 219 Setswana and 491 Sepedi news headlines from SABC radio stations and social media.
• Categorized headlines into 10 predefined topics: Legal, General News, Sports, Other, Politics, Traffic News, Community Activities, Crime, Business, and Foreign Affairs.
• Constructed multiple vectorizers using monolingual corpora: Bag of Words, TF-IDF, Word2Vec, and FastText for improved representation.
• Trained and evaluated multiple classifiers: Logistic Regression, Support Vector Classification, XGBoost, and Multi-layer Perceptron using 5-fold cross-validation.
• Applied word2vec-based data augmentation to enhance model generalization on low-resource text.
• Used pre-trained embeddings from Wikipedia, the Bible, and JW300 to improve vector representation for low-vocabulary datasets.

Experimental results

Research questions

• RQ1 Can a systematic framework for dataset creation and curation be established for low-resource languages like Setswana and Sepedi?
• RQ2 How effective are different vectorization techniques (e.g., TF-IDF, FastText) in representing short news headlines in low-resource settings?
• RQ3 To what extent does word2vec-based data augmentation improve classification performance on small, low-resource datasets?
• RQ4 What baseline performance can be achieved using traditional machine learning models on Setswana and Sepedi news headline classification?
• RQ5 How do different classifiers (e.g., XGBoost vs. MLP) compare in terms of accuracy and robustness on these low-resource datasets?

Key findings

• The study successfully created two annotated datasets with 219 Setswana and 491 Sepedi news headlines, covering diverse topics.
• Logistic Regression and XGBoost achieved competitive performance on both datasets, serving as strong baselines for future work.
• Data augmentation using word2vec improved classifier generalization, particularly in low-vocabulary settings.
• Pre-trained vectorizers built from monolingual corpora (e.g., Wikipedia, Bible) enhanced representation learning for short text.
• The framework demonstrated feasibility for extending similar approaches to other low-resource South African languages.
• The results highlight the importance of curated data and tailored vectorization for advancing NLP in under-resourced languages.