[Paper Review] StockEmotions: Discover Investor Emotions for Financial Sentiment Analysis and Multivariate Time Series

Jean Lee, Hoyoul Luis Youn+2 more|arXiv (Cornell University)|Jan 23, 2023

Stock Market Forecasting MethodsDecision Sciences3 citations

TL;DR

This paper introduces StockEmotions, a novel 10,000-sample dataset of financial social media comments from StockTwits, annotated with 12 fine-grained emotion classes and financial sentiment, enabling advanced sentiment and emotion analysis. Using a hybrid human-PLM annotation pipeline, the study demonstrates that combining text, emotion, and price features in a Temporal Attention LSTM improves multivariate time series forecasting performance on the S&P 500 beyond using price data alone.

ABSTRACT

There has been growing interest in applying NLP techniques in the financial domain, however, resources are extremely limited. This paper introduces StockEmotions, a new dataset for detecting emotions in the stock market that consists of 10,000 English comments collected from StockTwits, a financial social media platform. Inspired by behavioral finance, it proposes 12 fine-grained emotion classes that span the roller coaster of investor emotion. Unlike existing financial sentiment datasets, StockEmotions presents granular features such as investor sentiment classes, fine-grained emotions, emojis, and time series data. To demonstrate the usability of the dataset, we perform a dataset analysis and conduct experimental downstream tasks. For financial sentiment/emotion classification tasks, DistilBERT outperforms other baselines, and for multivariate time series forecasting, a Temporal Attention LSTM model combining price index, text, and emotion features achieves the best performance than using a single feature.

Motivation & Objective

• Address the scarcity of large-scale, fine-grained financial sentiment and emotion datasets in NLP for finance.
• Overcome limitations of existing datasets that lack emotion granularity, emoji, and time-series alignment.
• Develop a robust, multi-step annotation pipeline combining pre-trained language models and expert validation for emotion labeling.
• Demonstrate the utility of investor emotions in improving multivariate stock market time series forecasting.
• Establish baseline models for financial sentiment and emotion classification, and for joint forecasting using text, emotion, and price data.

Proposed method

• Collected 10,000 English comments from StockTwits, a financial social media platform, focusing on investor sentiment and emotional expression.
• Designed a 12-class emotion taxonomy grounded in behavioral finance and psychological research, including emotions like excitement, anxiety, anger, and optimism.
• Employed a hybrid annotation pipeline: first using a pre-trained language model (PLM) to predict emotions, then validated and corrected by finance experts and human annotators.
• Integrated multiple data modalities: financial sentiment (bullish/bearish), fine-grained emotions, emojis, and time-series stock index data (e.g., S&P 500).
• Trained and evaluated seven baseline models—Logistic Regression, NBSVM, GRU, Bi-GRU, DistilBERT, BERT, and RoBERTa—for sentiment and emotion classification.
• Built a Temporal Attention LSTM model that jointly processes price index, textual sentiment, and emotion features to forecast S&P 500 movements with attention over time windows.

Figure 1: Example from StockEmotions dataset showing investor psychology on the stock market. A combination of input data (stock price index, text and emoji, and emotion label) is used on a Temporal Attention LSTM for multivariate time series forecasting. (blue line = the actual S&P index, orange li

Experimental results

Research questions

• RQ1 Can a fine-grained, 12-class emotion taxonomy improve financial sentiment and emotion classification beyond binary sentiment labels?
• RQ2 How effective is a hybrid human-PLM annotation pipeline in generating reliable emotion labels for financial text?
• RQ3 To what extent do investor emotions enhance multivariate time series forecasting performance when combined with price and text data?
• RQ4 Which deep learning architecture—DistilBERT, BERT, or LSTM—yields the best performance for financial sentiment and emotion classification?
• RQ5 Does joint modeling of text, emotion, and price features outperform models using only numeric or textual inputs in S&P 500 forecasting?

Key findings

• DistilBERT achieved the highest F1-score of 0.81 for financial sentiment classification and 0.42 for emotion classification, outperforming other baselines.
• The Temporal Attention LSTM model that jointly learns from price index, text, and emotion features achieved the best forecasting performance on the S&P 500 compared to models using only price data.
• Emotion classification performance was moderate, with macro F1 scores of 0.48 (Ekman mapping) and 0.40 (Plutchik mapping) using BERT, indicating room for improvement in emotion detection in financial text.
• The dataset revealed high prevalence of financial slang and sarcasm, which confused models—e.g., 'BTD' (Buy The Dip) and 'ATH' (All Time High) were frequently misclassified.
• Emotion labels were often inconsistent with sentiment due to investor position bias: e.g., anger expressed by a long holder during a drop was labeled as 'bullish' sentiment, reflecting underlying optimism.
• The model's performance was sensitive to hyperparameter tuning, with learning rate and dropout values significantly affecting results, especially for smaller models like Logistic Regression and NBSVM.

Figure 2: An overview of dataset creation pipeline.