[论文解读] Improving the Accuracy of Pre-trained Word Embeddings for Sentiment Analysis

Seyed Mahdi Rezaeinia, Ali Ghodsi+1 more|arXiv (Cornell University)|Nov 23, 2017

Sentiment Analysis and Opinion Mining参考文献 24被引用 61

一句话总结

该论文提出了 Improved Word Vectors (IWV)，一种通过利用 POS 标注、词汇表线索，以及现有 Word2Vec/GloVe 表嵌入来提升情感分析的预训练词向量方法，并在神经网络模型和数据集上展示了改进的性能。

ABSTRACT

Sentiment analysis is one of the well-known tasks and fast growing research areas in natural language processing (NLP) and text classifications. This technique has become an essential part of a wide range of applications including politics, business, advertising and marketing. There are various techniques for sentiment analysis, but recently word embeddings methods have been widely used in sentiment classification tasks. Word2Vec and GloVe are currently among the most accurate and usable word embedding methods which can convert words into meaningful vectors. However, these methods ignore sentiment information of texts and need a huge corpus of texts for training and generating exact vectors which are used as inputs of deep learning models. As a result, because of the small size of some corpuses, researcher often have to use pre-trained word embeddings which were trained on other large text corpus such as Google News with about 100 billion words. The increasing accuracy of pre-trained word embeddings has a great impact on sentiment analysis research. In this paper we propose a novel method, Improved Word Vectors (IWV), which increases the accuracy of pre-trained word embeddings in sentiment analysis. Our method is based on Part-of-Speech (POS) tagging techniques, lexicon-based approaches and Word2Vec/GloVe methods. We tested the accuracy of our method via different deep learning models and sentiment datasets. Our experiment results show that Improved Word Vectors (IWV) are very effective for sentiment analysis.

研究动机与目标

• 激发需要改进情感分析的动机，使用可能缺乏情感信息的预训练词嵌入。
• 提出 IWV，将 POS 标注、基于词汇表的线索与现有嵌入相结合，以提升情感表示。
• 在多种深度学习模型和情感数据集上评估 IWV，以评估其泛化性。

提出的方法

• 通过基于 POS 标注的调整来增强预训练嵌入，从而开发 IWV。
• 融入基于词汇表的情感线 cues 以细化词表示。
• 将现有的 Word2Vec 和 GloVe 向量作为基础嵌入空间。
• 评估 IWV 对下游情感分析模型的影响。
• 尝试不同的深度学习架构以测试鲁棒性。
• 分析在多样化情感数据集上的性能。

实验结果

研究问题

• RQ1 与标准预训练嵌入相比，IWV 是否能提升情感分类准确性？
• RQ2 POS 标注与词汇表线索如何促进 IWV 的有效性？
• RQ3 IWV 是否在不同的神经网络架构和情感数据集上表现鲁棒？
• RQ4 将 IWV 与 Word2Vec/GloVe 结合对性能的总体影响是什么？

主要发现

• IWV 在所测试的模型中对情感分析有效。
• 将 POS 和词汇表线索与预训练嵌入结合可提升性能。
• 在各数据集上与 Word2Vec 和 GloVe 嵌入一起使用时，IWV 显示出改进。