[Paper Review] Sentiment Analysis with R: Natural Language Processing for Semi-Automated Assessments of Qualitative Data

Dennis Klinkhammer|arXiv (Cornell University)|Jun 25, 2022

Computational and Text Analysis Methods4 citations

TL;DR

This tutorial presents a practical, reproducible workflow for semi-automated sentiment analysis in R using natural language processing (NLP) techniques, enabling researchers to efficiently classify sentiment and emotional states in qualitative text data—such as social media comments, political speeches, and books—by leveraging sentiment lexicons and R packages like tidytext, dplyr, and ggplot2, with demonstrated results showing consistent inter-rater reliability and detectable shifts in sentiment across discourse over time.

ABSTRACT

Sentiment analysis is a sub-discipline in the field of natural language processing and computational linguistics and can be used for automated or semi-automated analyses of text documents. One of the aims of these analyses is to recognize an expressed attitude as positive or negative as it can be contained in comments on social media platforms or political documents and speeches as well as fictional and nonfictional texts. Regarding analyses of comments on social media platforms, this is an extension of the previous tutorial on semi-automated screenings of social media network data. A longitudinal perspective regarding social media comments as well as cross-sectional perspectives regarding fictional and nonfictional texts, e.g. entire books and libraries, can lead to extensive text documents. Their analyses can be simplified and accelerated by using sentiment analysis with acceptable inter-rater reliability. Therefore, this tutorial introduces the basic functions for performing a sentiment analysis with R and explains how text documents can be analysed step by step - regardless of their underlying formatting. All prerequisites and steps are described in detail and associated codes are available on GitHub. A comparison of two political speeches illustrates a possible use case.

Motivation & Objective

• To provide a reproducible, step-by-step tutorial for conducting sentiment analysis on qualitative text data using R.
• To enable researchers to apply both basic and advanced sentiment analysis techniques to diverse text formats, including social media, political speeches, and books.
• To demonstrate how sentiment analysis can enhance the efficiency and reliability of qualitative data coding with acceptable inter-rater reliability.
• To support methodological transparency by providing full R code and package dependencies via GitHub.
• To illustrate practical use cases, such as detecting shifts in sentiment and emotional states in political discourse over time.

Proposed method

• Utilizes R Markdown and the R programming language as the primary environment for sentiment analysis.
• Employs the tidytext package to tokenize and process text data into manageable units for analysis.
• Applies sentiment lexicons—specifically the NRC Word-Emotion Association Lexicon—to classify words into sentiment polarities (positive, negative) and eight emotional states (e.g., fear, anger, trust, joy).
• Uses dplyr for data manipulation, including filtering, grouping, and summarizing sentiment scores across text segments.
• Visualizes sentiment trends using ggplot2 to track sentiment distribution, conditional means, and score changes over time or text sections.
• Processes text by converting non-TXT formats (e.g., PDF, HTML) into TXT format for consistent preprocessing before analysis.

Experimental results

Research questions

• RQ1 How can sentiment analysis with R be used to semi-automatically assess sentiment and emotional states in diverse qualitative text data, such as political speeches and social media comments?
• RQ2 To what extent does semi-automated sentiment analysis achieve acceptable inter-rater reliability compared to manual coding?
• RQ3 How do sentiment and emotional state distributions shift across different sections of a political speech, and what do these shifts indicate?
• RQ4 Can sentiment analysis detect meaningful patterns in long-form texts like books or entire libraries, and how does this compare to short-form content like social media posts?
• RQ5 What is the practical utility of sentiment analysis in identifying social issues such as radicalization, political sentiment, or public health concerns in digital discourse?

Key findings

• Semi-automated sentiment analysis achieves inter-rater reliability of up to 70%, which is considered acceptable given that even manual coding shows around 20% disagreement among researchers.
• A comparison of two political speeches revealed that while positive and negative sentiments were used relatively equally, fear and anger were prominent, indicating a negative overall sentiment tone.
• The emotional state of trust was found to counterbalance negative emotions like fear and anger, suggesting a complex emotional landscape in political discourse.
• Sentiment scores showed a slight increase in the central portion of a political speech, indicating a possible strategic shift in tone or appeal.
• The method successfully identified turning points and saddle points in sentiment trends, enabling researchers to inspect key lines or paragraphs for deeper qualitative validation.
• The workflow significantly accelerates the analysis of large text corpora while maintaining analytical rigor, making it suitable for both longitudinal and cross-sectional text analysis.