[Paper Review] Uzbek Sentiment Analysis based on local Restaurant Reviews

Sanatbek Matlatipov, Hulkar Rahimboeva+2 more|arXiv (Cornell University)|May 31, 2022

Sentiment Analysis and Opinion Mining4 citations

TL;DR

This paper presents a novel, manually curated dataset of 8,210 Uzbek-language restaurant reviews from Google Maps in Tashkent, annotated for sentiment using star ratings (4–5 stars = positive, 1–3 = negative). It evaluates multiple models, with logistic regression using combined word and character n-grams achieving the highest accuracy of 91%, demonstrating the effectiveness of pre-processing, especially stemming, for low-resource agglutinative languages like Uzbek.

ABSTRACT

Extracting useful information for sentiment analysis and classification problems from a big amount of user-generated feedback, such as restaurant reviews, is a crucial task of natural language processing, which is not only for customer satisfaction where it can give personalized services, but can also influence the further development of a company. In this paper, we present a work done on collecting restaurant reviews data as a sentiment analysis dataset for the Uzbek language, a member of the Turkic family which is heavily affected by the low-resource constraint, and provide some further analysis of the novel dataset by evaluation using different techniques, from logistic regression based models, to support vector machines, and even deep learning models, such as recurrent neural networks, as well as convolutional neural networks. The paper includes detailed information on how the data was collected, how it was pre-processed for better quality optimization, as well as experimental setups for the evaluation process. The overall evaluation results indicate that by performing pre-processing steps, such as stemming for agglutinative languages, the system yields better results, eventually achieving 91% accuracy result in the best performing model

Motivation & Objective

• To address the lack of sentiment analysis resources for low-resource, agglutinative languages like Uzbek by creating a domain-specific dataset.
• To evaluate the performance of diverse machine learning and deep learning models on a newly collected Uzbek restaurant review dataset.
• To investigate the impact of pre-processing steps—especially stemming and stopword removal—on model accuracy in low-resource settings.
• To provide a publicly available dataset and codebase to support future NLP research in Uzbek and similar languages.

Proposed method

• The dataset was collected via web crawling from Google Maps using URLs of restaurants in Tashkent, focusing on local Uzbek cuisine.
• Reviews were annotated using the 5-star rating system: 4–5 stars labeled as positive, 1–3 as negative, with 1,000+ reviews translated into Uzbek using Google Translate API.
• Pre-processing included removing URLs, punctuation, and case conversion, followed by stopword removal using TF-IDF-derived stopword lists.
• A custom morphological stemmer based on Uzbek word endings was applied to handle agglutinative morphology, enabling better segmentation and analysis.
• Multiple models were evaluated: logistic regression (with word and character n-grams), SVM with linear kernel, RNN, and CNN, with and without fastText word embeddings.
• Evaluation used standard metrics: precision, recall, F1-score, and accuracy, with cross-validation planned for future work to reduce bias.

Experimental results

Research questions

• RQ1 How effective are traditional machine learning models like logistic regression and SVM on a low-resource, agglutinative language like Uzbek for sentiment analysis?
• RQ2 To what extent does pre-processing, particularly stemming and stopword removal, improve sentiment classification accuracy in Uzbek?
• RQ3 How do deep learning models such as RNNs and CNNs perform on a relatively small Uzbek sentiment dataset compared to classical models?
• RQ4 Can pre-trained word embeddings (e.g., fastText) improve performance on a low-resource language like Uzbek?
• RQ5 How does the imbalance between positive and negative reviews affect model performance and generalization?

Key findings

• The logistic regression model using both word and character n-grams achieved the highest accuracy of 91%, outperforming other models.
• The model using only character n-grams achieved 90% accuracy, showing improved robustness to spelling variations.
• SVM with a linear kernel achieved 88% accuracy, while RNN and CNN models achieved 88% and 89.23% respectively, indicating limited performance gains from deep learning due to dataset size.
• The use of a custom stemmer based on Uzbek word endings significantly improved model performance by enabling proper morphological analysis.
• Stopword removal using TF-IDF improved model accuracy, confirming the relevance of language-specific stopword lists in low-resource settings.
• The dataset contains 4,500 positive and 3,710 negative reviews after manual cleaning, with a total of 8,210 annotated reviews.