[Paper Review] Twitter Sentiment Analysis: How To Hedge Your Bets In The Stock Markets

Tushar Rao, Saket Srivastava|arXiv (Cornell University)|Dec 5, 2012

Stock Market Forecasting Methods4 citations

TL;DR

This paper proposes a scalable Twitter sentiment analysis model that correlates public mood from microblogging data with short-term stock market movements, using Granger causality and an Expert Model Mining System (EMMS) to forecast DJIA returns with 90.8% directional accuracy and an R-squared of 0.952. The approach enables improved hedging strategies by dynamically adjusting portfolios based on sentiment-driven signals.

ABSTRACT

Emerging interest of trading companies and hedge funds in mining social web has created new avenues for intelligent systems that make use of public opinion in driving investment decisions. It is well accepted that at high frequency trading, investors are tracking memes rising up in microblogging forums to count for the public behavior as an important feature while making short term investment decisions. We investigate the complex relationship between tweet board literature (like bullishness, volume, agreement etc) with the financial market instruments (like volatility, trading volume and stock prices). We have analyzed Twitter sentiments for more than 4 million tweets between June 2010 and July 2011 for DJIA, NASDAQ-100 and 11 other big cap technological stocks. Our results show high correlation (upto 0.88 for returns) between stock prices and twitter sentiments. Further, using Granger's Causality Analysis, we have validated that the movement of stock prices and indices are greatly affected in the short term by Twitter discussions. Finally, we have implemented Expert Model Mining System (EMMS) to demonstrate that our forecasted returns give a high value of R-square (0.952) with low Maximum Absolute Percentage Error (MaxAPE) of 1.76% for Dow Jones Industrial Average (DJIA). We introduce a novel way to make use of market monitoring elements derived from public mood to retain a portfolio within limited risk state (highly improved hedging bets) during typical market conditions.

Motivation & Objective

• To investigate the causal relationship between Twitter sentiment and short-term financial market movements in major indices and tech stocks.
• To develop a scalable, customizable method for measuring public sentiment toward specific companies or indices using microblogging data.
• To validate that Twitter sentiment features—such as bullishness, agreement, and volume—Granger-cause stock returns and volatility.
• To design and implement an Expert Model Mining System (EMMS) that enables real-time, sentiment-based portfolio hedging strategies.
• To demonstrate the practical utility of sentiment data in reducing investment risk through dynamic, sentiment-driven portfolio adjustments.

Proposed method

• Collected over 4 million tweets from June 2010 to July 2011 using Twitter’s search API, focusing on DJIA, NASDAQ-100, and 11 big-cap tech stocks.
• Defined sentiment features including bullishness (positive sentiment), agreement (consensus on sentiment), and volume (discussion intensity) for each company/index.
• Applied Granger Causality Analysis (GCA) to test whether lagged Twitter sentiment features predict future stock returns, volatility, and trading volume.
• Employed Correlation Analysis to measure the strength of linear relationships between sentiment features and financial indicators.
• Implemented an Expert Model Mining System (EMMS) to forecast stock returns using sentiment-based inputs, optimizing for R-squared and Maximum Absolute Percentage Error (MaxAPE).
• Validated model performance using out-of-sample testing on DJIA and NASDAQ-100 data, comparing directional accuracy and error metrics.

Experimental results

Research questions

• RQ1 To what extent do Twitter sentiment features (bullishness, agreement, volume) Granger-cause short-term movements in stock prices and indices?
• RQ2 How do lagged sentiment features (1–3 weeks) predict future returns for DJIA and NASDAQ-100?
• RQ3 Can a sentiment-based model outperform previous approaches in directional accuracy and predictive R-squared for stock market indices?
• RQ4 How can Twitter sentiment be used to dynamically adjust portfolio positions to reduce risk in volatile market conditions?
• RQ5 What is the scalability and customizability of sentiment modeling for individual stocks versus broad indices?

Key findings

• A strong correlation (up to 0.88) was found between Twitter sentiment features and stock returns for individual companies, particularly in the tech sector.
• Granger Causality Analysis confirmed that Twitter sentiment features significantly predict future returns, with the strongest predictive power at a 2-week lag.
• The Expert Model Mining System (EMMS) achieved an R-squared of 0.952 and a MaxAPE of 1.76% for forecasting DJIA returns, outperforming prior models.
• The model demonstrated 90.8% directional accuracy in predicting DJIA movements, significantly higher than previous studies (e.g., 86.7% in Bollen et al.).
• Sentiment features such as agreement and volume were found to be more predictive than raw sentiment polarity alone, especially in detecting market turning points.
• The approach enables effective dynamic hedging by triggering portfolio rebalancing (e.g., bearish to bullish) based on sentiment shifts, reducing exposure during market downturns.