[论文解读] Bidirectional Encoder Representations from Transformers (BERT): A sentiment analysis odyssey

Shivaji Alaparthi, Manit Mishra|arXiv (Cornell University)|Jul 2, 2020

Sentiment Analysis and Opinion Mining参考文献 25被引用 53

一句话总结

本文比较了四种情感分析技术（Sent WordNet、逻辑回归、LSTM 和 BERT）在 IMDB 影评上的表现，并显示 BERT 的性能更优。

ABSTRACT

The purpose of the study is to investigate the relative effectiveness of four different sentiment analysis techniques: (1) unsupervised lexicon-based model using Sent WordNet; (2) traditional supervised machine learning model using logistic regression; (3) supervised deep learning model using Long Short-Term Memory (LSTM); and, (4) advanced supervised deep learning models using Bidirectional Encoder Representations from Transformers (BERT). We use publicly available labeled corpora of 50,000 movie reviews originally posted on internet movie database (IMDB) for analysis using Sent WordNet lexicon, logistic regression, LSTM, and BERT. The first three models were run on CPU based system whereas BERT was run on GPU based system. The sentiment classification performance was evaluated based on accuracy, precision, recall, and F1 score. The study puts forth two key insights: (1) relative efficacy of four highly advanced and widely used sentiment analysis techniques; (2) undisputed superiority of pre-trained advanced supervised deep learning BERT model in sentiment analysis from text data. This study provides professionals in analytics industry and academicians working on text analysis key insight regarding comparative classification performance evaluation of key sentiment analysis techniques, including the recently developed BERT. This is the first research endeavor to compare the advanced pre-trained supervised deep learning model of BERT vis-à-vis other sentiment analysis models of LSTM, logistic regression, and Sent WordNet.

研究动机与目标

• 在同一数据集上评估并比较四种情感分析技术。
• 使用标准指标（准确率、精确率、召回率、F1）评估性能。
• 突出预训练 BERT 相对于传统模型的相对有效性。

提出的方法

• 使用 Sent WordNet 进行无监督的词典基情感分析。
• 在带标签的数据上使用逻辑回归训练传统的监督模型。
• 训练基于 LSTM 的监督深度学习模型。
• 在相同数据上进行预训练 BERT 模型的微调并使用 GPU 加速进行评估。

实验结果

研究问题

• RQ1 哪种情感分析方法在 IMDB 影评上能达到最高的准确率、精确率、召回率和 F1？
• RQ2 像 BERT 这样的预训练高级模型在情感分析中是否始终优于传统和其他深度学习方法？
• RQ3 计算资源（CPU 与 GPU）如何影响模型间的性能比较？

主要发现

• 在本研究中，BERT 在四种技术的情感分析中显示出无可争议的优势。
• 在 GPU 上评估的预训练 BERT 在 IMDB 语料库上优于 Sent WordNet、逻辑回归和 LSTM。
• 该研究提供了并列的性能比较，以便从业者了解各方法的相对优势。