[Paper Review] Zero-shot hashtag segmentation for multilingual sentiment analysis

Ruan Chaves Rodrigues, Marcelo Akira Inuzuka+4 more|arXiv (Cornell University)|Dec 6, 2021

Sentiment Analysis and Opinion Mining4 citations

TL;DR

This paper proposes a zero-shot hashtag segmentation framework using pretrained Transformer models (GPT-2 and BERT) with beam search and re-ranking, achieving state-of-the-art performance on hashtag segmentation without fine-tuning. It demonstrates that this approach significantly improves multilingual sentiment analysis by enabling accurate segmentation and translation of hashtags across low-resource languages, outperforming baseline translation methods in several languages.

ABSTRACT

Hashtag segmentation, also known as hashtag decomposition, is a common step in preprocessing pipelines for social media datasets. It usually precedes tasks such as sentiment analysis and hate speech detection. For sentiment analysis in medium to low-resourced languages, previous research has demonstrated that a multilingual approach that resorts to machine translation can be competitive or superior to previous approaches to the task. We develop a zero-shot hashtag segmentation framework and demonstrate how it can be used to improve the accuracy of multilingual sentiment analysis pipelines. Our zero-shot framework establishes a new state-of-the-art for hashtag segmentation datasets, surpassing even previous approaches that relied on feature engineering and language models trained on in-domain data.

Motivation & Objective

• To develop a zero-shot hashtag segmentation framework that avoids the need for task-specific fine-tuning or in-domain pretraining.
• To evaluate the effectiveness of general-purpose pretrained language models (e.g., GPT-2, BERT) in zero-shot hashtag segmentation.
• To integrate hashtag segmentation into multilingual sentiment analysis pipelines to improve performance across low-resource languages.
• To demonstrate that zero-shot segmentation can outperform or match previous methods that relied on feature engineering or in-domain fine-tuning.
• To release an open-source implementation for reproducibility and production use.

Proposed method

• Combines GPT-2 and BERT as Segmenter and Re-ranker, respectively, using beam search to generate candidate segmentations.
• Applies re-ranking with a learned scoring function using hyperparameters α=0.2 and β=0.1, optimized on development sets.
• Processes hashtags by segmenting them into word-like units using the Segmenter module, then re-ranks candidates based on fluency and coherence.
• Applies the framework to multilingual datasets using language-specific models from Table 5, ensuring cross-lingual compatibility.
• Integrates the segmentation step into multilingual sentiment analysis pipelines via three translation strategies: Translation (T), Code-Mixed Translation (CMT), and Code-Mixed Translation and Segmentation (CMTS).
• Uses MarianMT for automatic translation and evaluates performance using F-score on the UMSAB benchmark.

Experimental results

Research questions

• RQ1 Can general-purpose pretrained language models achieve competitive zero-shot hashtag segmentation performance without fine-tuning?
• RQ2 How does zero-shot hashtag segmentation improve multilingual sentiment analysis when integrated into translation pipelines?
• RQ3 Does the CMTS method, which combines segmentation and translation, consistently outperform direct translation (T) across diverse languages?
• RQ4 What is the impact of model architecture and pre-training data on zero-shot segmentation performance across low-resource languages?
• RQ5 Can a unified zero-shot framework replace the need for language-specific models in hashtag segmentation for sentiment analysis?

Key findings

• The proposed zero-shot framework achieves state-of-the-art results on the TEST-BOUN hashtag segmentation dataset, surpassing methods that rely on in-domain fine-tuning or feature engineering.
• In Arabic, the CMTS method achieved an F-score of 76.4%, a 3.3% improvement over the translation baseline (73.1%).
• For German and Italian, the CMTS method achieved notable gains over the baseline, demonstrating effectiveness in closely related languages.
• For Spanish, the CMTS method scored 1.0% below the baseline, indicating limited gains in some language pairs, likely due to model-specific pre-training characteristics.
• The Hindi dataset remained the most challenging, with no improvement observed over the baseline, highlighting limitations in low-resource language transfer.
• The framework successfully enables a multilingual sentiment analysis pipeline without requiring language-specific model training, reducing computational costs.