[Paper Review] Financial Time Series Forecasting using CNN and Transformer

Zhen Zeng, Rachneet Kaur+4 more|arXiv (Cornell University)|Apr 11, 2023

Stock Market Forecasting Methods23 citations

TL;DR

The paper proposes CTTS, a CNN-Transformer model that captures short-term and long-term dependencies to forecast intraday stock price movements, outperforming ARIMA, EMA, and DeepAR on S&P 500 data.

ABSTRACT

Time series forecasting is important across various domains for decision-making. In particular, financial time series such as stock prices can be hard to predict as it is difficult to model short-term and long-term temporal dependencies between data points. Convolutional Neural Networks (CNN) are good at capturing local patterns for modeling short-term dependencies. However, CNNs cannot learn long-term dependencies due to the limited receptive field. Transformers on the other hand are capable of learning global context and long-term dependencies. In this paper, we propose to harness the power of CNNs and Transformers to model both short-term and long-term dependencies within a time series, and forecast if the price would go up, down or remain the same (flat) in the future. In our experiments, we demonstrated the success of the proposed method in comparison to commonly adopted statistical and deep learning methods on forecasting intraday stock price change of S&P 500 constituents.

Motivation & Objective

• Motivate accurate forecasting of financial time series by capturing both short-term and long-term dependencies.
• Propose a CNN-Transformer based architecture to predict price movement directions (up/down/flat).
• Evaluate performance against statistical and deep learning baselines on real intraday stock data.

Proposed method

• Preprocess data with min-max scaling.
• Represent local short-term patterns as CNN-generated tokens via 1D convolutions.
• Incorporate positional embeddings and pass tokens through a Transformer to learn long-range dependencies.
• Encode Transformer output with an MLP and softmax to predict one of three classes (up, down, flat).
• Train using cross-entropy loss with AdamW on GPU hardware.

Experimental results

Research questions

• RQ1 Can a CNN-Transformer hybrid model effectively capture both short-term and long-term dependencies in financial time series to predict price movement direction?
• RQ2 Does CTTS outperform traditional statistical methods (ARIMA, EMA) and a deep learning autoregressive model (DeepAR) on intraday stock data?
• RQ3 Is the model's prediction probability reliable enough to enable threshold-based high-confidence trading decisions?

Key findings

• CTTS outperforms all baselines in sign accuracy for both 2-class and 3-class tasks.
• Thresholded prediction accuracy improves for all models, with CTTS showing the largest gains.
• CTTS achieves 56.7% (2-class), 66.8% (2-class ∗), 44.1% (3-class), and 55.2% (3-class ∗) sign accuracies, which are higher than ARIMA, EMA, and DeepAR.
• CTTS's probability estimates are reliable, enabling higher accuracy when applying a confidence threshold.
• The study uses intraday S&P 500 data from 2019 and demonstrates potential for trading decisions based on predicted classes.