[Paper Review] Stock and market index prediction using Informer network

Yuze Lu, Hailong Zhang+1 more|arXiv (Cornell University)|May 22, 2023

Stock Market Forecasting Methods8 citations

TL;DR

The paper introduces Informer, a Transformer-based network with ProbSparse self-attention, self-attention distilling, and global time stamp features to predict minute-scale stock prices, showing superior accuracy over LSTM, Transformer, and BERT across multiple datasets and demonstrating transfer learning capabilities.

ABSTRACT

Applications of deep learning in financial market prediction has attracted huge attention from investors and researchers. In particular, intra-day prediction at the minute scale, the dramatically fluctuating volume and stock prices within short time periods have posed a great challenge for the convergence of networks result. Informer is a more novel network, improved on Transformer with smaller computational complexity, longer prediction length and global time stamp features. We have designed three experiments to compare Informer with the commonly used networks LSTM, Transformer and BERT on 1-minute and 5-minute frequencies for four different stocks/ market indices. The prediction results are measured by three evaluation criteria: MAE, RMSE and MAPE. Informer has obtained best performance among all the networks on every dataset. Network without the global time stamp mechanism has significantly lower prediction effect compared to the complete Informer; it is evident that this mechanism grants the time series to the characteristics and substantially improves the prediction accuracy of the networks. Finally, transfer learning capability experiment is conducted, Informer also achieves a good performance. Informer has good robustness and improved performance in market prediction, which can be exactly adapted to real trading.

Motivation & Objective

• Motivate intraday stock price forecasting given highly volatile minute-scale volume and price changes.
• Evaluate Informer's performance against popular architectures (LSTM, Transformer, BERT) on 1- and 5-minute data.
• Explore the impact of global time stamp embedding on prediction accuracy.
• Demonstrate Transfer Learning capability of Informer across markets, time scales, and assets.

Proposed method

• Adopt Informer, an improved Transformer with ProbSparse self-attention to reduce complexity from O(L^2) to O(L log L).
• Incorporate self-attention distilling to progressively reduce sequence length and save memory.
• Use a generative inference approach to output entire future sequences at once with a guiding sequence.
• Add a global time stamp embedding to capture minute-level temporal context (year, month, week, hour, minute).
• Train and compare Informer against LSTM, Transformer, and BERT using 1-minute and 5-minute data from HSI, IXIC, Tencent, and AAPL.
• Evaluate using MAE, RMSE, and MAPE with 70/10/20 train/validation/test splits.

Experimental results

Research questions

• RQ1 Does Informer outperform LSTM, Transformer, and BERT for minute-scale stock and market index prediction?
• RQ2 How does ProbSparse self-attention affect predictive accuracy and efficiency for short-time horizons?
• RQ3 What is the impact of introducing a global time stamp on forecasting performance in volatile intraday periods?
• RQ4 Can Informer demonstrate transfer learning effectiveness across different markets, assets, and time scales?

Key findings

• Informer achieves the best performance across all datasets and time scales for MAE, RMSE, and MAPE compared with LSTM, Transformer, and BERT.
• Global time stamp embedding significantly improves prediction accuracy in volatile, intraday periods compared to using only positional embeddings.
• Removing the global time stamp (Informer�) degrades performance, especially in datasets with active trading.
• Informer demonstrates transfer learning capability with reasonable prediction accuracy when weights pre-trained on AAPL are applied to HSI, IXIC, and Tencent, despite cross-market differences.
• Overall, Informer shows robustness and superior short-term forecasting suitable for real trading.