[Paper Review] HiSA-SMFM: Historical and Sentiment Analysis based Stock Market Forecasting Model

Ishu Gupta, Tarun Kumar Madan+2 more|arXiv (Cornell University)|Mar 10, 2022

Stock Market Forecasting Methods27 citations

TL;DR

HiSA-SMFM integrates historical stock data with Twitter sentiment using TextBlob and Tweepy, trained via LSTM, to improve stock price prediction on Tata Motors (NSE).

ABSTRACT

One of the pillars to build a country's economy is the stock market. Over the years, people are investing in stock markets to earn as much profit as possible from the amount of money that they possess. Hence, it is vital to have a prediction model which can accurately predict future stock prices. With the help of machine learning, it is not an impossible task as the various machine learning techniques if modeled properly may be able to provide the best prediction values. This would enable the investors to decide whether to buy, sell or hold the share. The aim of this paper is to predict the future of the financial stocks of a company with improved accuracy. In this paper, we have proposed the use of historical as well as sentiment data to efficiently predict stock prices by applying LSTM. It has been found by analyzing the existing research in the area of sentiment analysis that there is a strong correlation between the movement of stock prices and the publication of news articles. Therefore, in this paper, we have integrated these factors to predict the stock prices more accurately.

Motivation & Objective

• Motivate improved stock price prediction by incorporating sentiment data alongside historical prices.
• Develop a model that can dynamically integrate multi-feature sentiment signals with historical data.
• Utilize LSTM to learn from combined historical and sentiment features for future price forecasting.
• Validate the model on a real Indian stock (Tata Motors) and compare against a state-of-the-art baseline.

Proposed method

• Use Tweepy to collect Twitter data for sentiment signals related to stocks.
• Apply TextBlob to classify sentiment into positive, negative, and neutral and convert to percentage features.
• Extract historical stock features (open price, etc.) from NSE India for Tata Motors.
• Merge sentiment percentages with historical features to form a multi-feature training set.
• Train an LSTM network on the integrated dataset to predict future stock prices.

Experimental results

Research questions

• RQ1 Does combining historical price data with Twitter-derived sentiment improve stock price prediction compared to using historical data alone?
• RQ2 How does the HiSA-SMFM model perform across different training epochs when predicting Tata Motors stock price?
• RQ3 What is the relative performance gain of HiSA-SMFM over the DLPM baseline when using multi-feature inputs?
• RQ4 Can a multi-feature sentiment representation (positive/negative percentages) be effectively integrated into LSTM forecasting?

Key findings

• HiSA-SMFM achieves higher accuracy than the DLPM baseline across epoch sizes 5, 10, and 15.
• At epoch 5, DLPM: 91.59% vs HiSA-SMFM: 95.41%.
• At epoch 10, DLPM: 94.56% vs HiSA-SMFM: 97.18%.
• At epoch 15, DLPM: 83.46% vs HiSA-SMFM: 92.38%.
• Average accuracy: DLPM 89.87% vs HiSA-SMFM 94.99%, showing >5% improvement for HiSA-SMFM.