[Paper Review] Multi-channel CNN to classify nepali covid-19 related tweets using hybrid features

Chiranjibi Sitaula, Tej Bahadur Shahi|arXiv (Cornell University)|Mar 19, 2022

Misinformation and Its Impacts24 references4 citations

TL;DR

This paper proposes a multi-channel CNN (MCNN) with hybrid feature extraction—combining bag-of-words (syntactic) and fastText/domain-specific embeddings (semantic)—to classify Nepali COVID-19 tweets into positive, neutral, and negative sentiments. The MCNN model achieves a 71.3% classification accuracy on the NepCOV19Tweets dataset, outperforming existing methods by effectively capturing multi-scale features through channel-wise learning and ensemble fusion.

ABSTRACT

Because of the current COVID-19 pandemic with its increasing fears among people, it has triggered several health complications such as depression and anxiety. Such complications have not only affected the developed countries but also developing countries such as Nepal. These complications can be understood from peoples' tweets/comments posted online after their proper analysis and sentiment classification. Nevertheless, owing to the limited number of tokens/words in each tweet, it is always crucial to capture multiple information associated with them for their better understanding. In this study, we, first, represent each tweet by combining both syntactic and semantic information, called hybrid features. The syntactic information is generated from the bag of words method, whereas the semantic information is generated from the combination of the fastText-based (ft) and domain-specific (ds) methods. Second, we design a novel multi-channel convolutional neural network (MCNN), which ensembles the multiple CNNs, to capture multi-scale information for better classification. Last, we evaluate the efficacy of both the proposed feature extraction method and the MCNN model classifying tweets into three sentiment classes (positive, neutral and negative) on NepCOV19Tweets dataset, which is the only public COVID-19 tweets dataset in Nepali language. The evaluation results show that the proposed hybrid features outperform individual feature extraction methods with the highest classification accuracy of 69.7% and the MCNN model outperforms the existing methods with the highest classification accuracy of 71.3% during classification.

Motivation & Objective

• Address the challenge of sentiment classification in low-resource languages like Nepali, where traditional NLP methods are limited by short text length and lack of contextual features.
• Overcome the limitations of single-feature approaches (e.g., bag-of-words or word embeddings alone) by combining syntactic and semantic information for improved tweet representation.
• Develop a novel multi-channel CNN architecture that captures multi-scale discriminative features from hybrid features for enhanced classification performance.
• Demonstrate the effectiveness of the proposed method on a publicly available Nepali COVID-19 tweet dataset, which is the only such dataset in the Nepali language.
• Establish a robust baseline for future NLP research in Nepali by integrating domain-specific and pre-trained embeddings with deep learning models.

Proposed method

• Construct hybrid features by combining bag-of-words (syntactic) with fastText-based and domain-specific word embeddings (semantic) to enrich tweet representation.
• Design a multi-channel CNN (MCNN) with four parallel convolutional branches: three for individual feature types (BoW, fastText, domain-specific) and one for concatenated hybrid features.
• Apply 1D convolutional layers with ReLU activation and max-pooling to extract local, multi-scale features from each channel, followed by global average pooling.
• Fuse the outputs of the four channels using an average-based decision fusion strategy to balance high and low confidence predictions.
• Train the MCNN model end-to-end using stochastic gradient descent with categorical cross-entropy loss and Adam optimizer.
• Evaluate model performance using 10-fold cross-validation and statistical significance testing (p-value < 2.2e-16) to ensure robustness.

Experimental results

Research questions

• RQ1 Can hybrid feature extraction combining syntactic and semantic information improve sentiment classification performance on low-resource Nepali tweets compared to individual feature types?
• RQ2 Does a multi-channel CNN architecture that processes multiple feature types in parallel outperform single-channel models in classifying Nepali COVID-19 tweets?
• RQ3 Which decision fusion strategy (sum, max, avg) yields the most stable and accurate classification results when combining predictions from multiple CNN channels?
• RQ4 How does the MCNN model perform across different sentiment classes (positive, neutral, negative) in terms of precision, recall, and F1-score?
• RQ5 Is the proposed method statistically significant and robust across multiple cross-validation folds?

Key findings

• The proposed hybrid feature extraction method achieved the highest classification accuracy of 69.7%, outperforming individual feature types (BoW, fastText, domain-specific) in sentiment classification.
• The MCNN model achieved a state-of-the-art classification accuracy of 71.3%, surpassing existing methods on the NepCOV19Tweets dataset.
• The average-based decision fusion strategy produced the most stable performance, preserving both high and low confidence predictions across classes.
• The MCNN model achieved a precision of 73.3%, recall of 75.1%, and F1-score of 74.2% for the negative class, with 68.8% precision and 83.3% recall for the positive class.
• Statistical analysis confirmed the model's robustness, with a 95% confidence interval of [71.0, 71.5] for accuracy and p-value < 2.2e-16 across all metrics.
• Ablation studies showed that each individual channel contributed discriminative features, with channel C3 achieving the highest F1-score (75.4%) for positive tweets, and the full MCNN outperforming all single-channel models in all metrics.