[Paper Review] Improved Twitter Sentiment Analysis Using Naive Bayes and Custom Language Model

Angela S. Lin|arXiv (Cornell University)|Nov 10, 2017

Sentiment Analysis and Opinion Mining2 references3 citations

TL;DR

This paper proposes a hybrid approach combining Naive Bayes classification with a custom-built natural language model to improve Twitter sentiment analysis, specifically targeting promotional content in developer communities. The method achieves 90% accuracy in identifying product-specific promotions, enhancing user contribution scoring for businesses like Microsoft Azure.

ABSTRACT

In the last couple decades, social network services like Twitter have generated large volumes of data about users and their interests, providing meaningful business intelligence so organizations can better understand and engage their customers. All businesses want to know who is promoting their products, who is complaining about them, and how are these opinions bringing or diminishing value to a company. Companies want to be able to identify their high-value customers and quantify the value each user brings. Many businesses use social media metrics to calculate the user contribution score, which enables them to quantify the value that influential users bring on social media, so the businesses can offer them more differentiated services. However, the score calculation can be refined to provide a better illustration of a user's contribution. Using Microsoft Azure as a case study, we conducted Twitter sentiment analysis to develop a machine learning classification model that identifies tweet contents and sentiments most illustrative of positive-value user contribution. Using data mining and AI-powered cognitive tools, we analyzed factors of social influence and specifically, promotional language in the developer community. Our predictive model was a combination of a traditional supervised machine learning algorithm and a custom-developed natural language model for identifying promotional tweets, that identifies a product-specific promotion on Twitter with a 90% accuracy rate.

Motivation & Objective

• To enhance user contribution scoring in social media by refining sentiment analysis for business intelligence.
• To identify promotional language in Twitter posts from developer communities to quantify user value.
• To develop a machine learning model that combines traditional classification with a custom language model for improved accuracy.
• To apply the model to real-world data from Microsoft Azure’s developer community for actionable insights.

Proposed method

• The study employs a supervised Naive Bayes classifier trained on labeled Twitter data to classify sentiment and content type.
• A custom natural language model is developed to detect promotional language, particularly product-specific mentions.
• The two models are combined to improve classification performance, especially in identifying high-value promotional content.
• Data mining and AI-powered cognitive tools are used to extract features related to social influence and promotional intent.
• The model is evaluated on real Twitter data from Microsoft Azure’s developer community.
• The final predictive model integrates both traditional machine learning and domain-specific linguistic patterns for better accuracy.

Experimental results

Research questions

• RQ1 Can a hybrid model of Naive Bayes and a custom language model improve the detection of promotional tweets on Twitter?
• RQ2 How accurately can such a model identify product-specific promotions in developer communities?
• RQ3 To what extent does incorporating domain-specific linguistic patterns enhance sentiment analysis for user contribution scoring?
• RQ4 Can this approach provide a more refined metric for quantifying user value on social media platforms?

Key findings

• The proposed model achieves a 90% accuracy rate in identifying product-specific promotions on Twitter.
• The integration of a custom language model significantly improves the detection of promotional content beyond standard Naive Bayes alone.
• The model successfully captures promotional language in developer communities, enabling better quantification of user contributions.
• The approach provides a more accurate and actionable user contribution score for businesses using social media data.