[Paper Review] Improving the Accuracy of Pre-trained Word Embeddings for Sentiment Analysis

Seyed Mahdi Rezaeinia, Ali Ghodsi+1 more|arXiv (Cornell University)|Nov 23, 2017

Sentiment Analysis and Opinion Mining24 references61 citations

TL;DR

The paper introduces Improved Word Vectors (IWV), a method that enhances pre-trained word embeddings for sentiment analysis by leveraging POS tagging, lexicon-based cues, and existing Word2Vec/GloVe embeddings, and demonstrates improved performance across neural models and datasets.

ABSTRACT

Sentiment analysis is one of the well-known tasks and fast growing research areas in natural language processing (NLP) and text classifications. This technique has become an essential part of a wide range of applications including politics, business, advertising and marketing. There are various techniques for sentiment analysis, but recently word embeddings methods have been widely used in sentiment classification tasks. Word2Vec and GloVe are currently among the most accurate and usable word embedding methods which can convert words into meaningful vectors. However, these methods ignore sentiment information of texts and need a huge corpus of texts for training and generating exact vectors which are used as inputs of deep learning models. As a result, because of the small size of some corpuses, researcher often have to use pre-trained word embeddings which were trained on other large text corpus such as Google News with about 100 billion words. The increasing accuracy of pre-trained word embeddings has a great impact on sentiment analysis research. In this paper we propose a novel method, Improved Word Vectors (IWV), which increases the accuracy of pre-trained word embeddings in sentiment analysis. Our method is based on Part-of-Speech (POS) tagging techniques, lexicon-based approaches and Word2Vec/GloVe methods. We tested the accuracy of our method via different deep learning models and sentiment datasets. Our experiment results show that Improved Word Vectors (IWV) are very effective for sentiment analysis.

Motivation & Objective

• Motivate the need to improve sentiment analysis using pre-trained word embeddings that may lack sentiment information.
• Propose IWV, combining POS tagging, lexicon-based cues, and existing embeddings to boost sentiment representation.
• Evaluate IWV across multiple deep learning models and sentiment datasets to assess generalizability.

Proposed method

• Develop IWV by augmenting pre-trained embeddings with POS-tag based adjustments.
• Incorporate lexicon-based sentiment cues to refine word representations.
• Leverage existing Word2Vec and GloVe vectors as the base embedding space.
• Assess the impact of IWV on downstream sentiment analysis models.
• Experiment with different deep learning architectures to test robustness.
• Analyze performance across diverse sentiment datasets.

Experimental results

Research questions

• RQ1 Does IWV improve sentiment classification accuracy compared to standard pre-trained embeddings?
• RQ2 How do POS tagging and lexicon cues contribute to IWV’s effectiveness?
• RQ3 Is IWV robust across different neural architectures and sentiment datasets?
• RQ4 What is the overall impact of combining IWV with Word2Vec/GloVe on performance?

Key findings

• IWV is effective for sentiment analysis across tested models.
• Incorporating POS and lexicon cues with pre-trained embeddings improves performance.
• IWV demonstrates improvements when used with Word2Vec and GloVe embeddings across datasets.