[Paper Review] Evaluating the Performance of Machine Learning Algorithms in Financial Market Forecasting: A Comprehensive Survey

Lukas Ryll, Sebastian Seidens|arXiv (Cornell University)|Jun 18, 2019

Stock Market Forecasting Methods21 citations

TL;DR

This paper conducts a comprehensive survey of over 150 studies on machine learning in financial market forecasting, evaluating algorithm performance across seven experimental parameters. It finds that machine learning models, particularly recurrent neural networks like LSTM and GRU, significantly outperform traditional stochastic models and feedforward networks due to their ability to capture exploitable temporal dependencies in financial time series across multiple asset classes and geographies.

ABSTRACT

With increasing competition and pace in the financial markets, robust forecasting methods are becoming more and more valuable to investors. While machine learning algorithms offer a proven way of modeling non-linearities in time series, their advantages against common stochastic models in the domain of financial market prediction are largely based on limited empirical results. The same holds true for determining advantages of certain machine learning architectures against others. This study surveys more than 150 related articles on applying machine learning to financial market forecasting. Based on a comprehensive literature review, we build a table across seven main parameters describing the experiments conducted in these studies. Through listing and classifying different algorithms, we also introduce a simple, standardized syntax for textually representing machine learning algorithms. Based on performance metrics gathered from papers included in the survey, we further conduct rank analyses to assess the comparative performance of different algorithm classes. Our analysis shows that machine learning algorithms tend to outperform most traditional stochastic methods in financial market forecasting. We further find evidence that, on average, recurrent neural networks outperform feed forward neural networks as well as support vector machines which implies the existence of exploitable temporal dependencies in financial time series across multiple asset classes and geographies.

Motivation & Objective

• To systematically evaluate the performance of machine learning algorithms in financial market forecasting across diverse studies.
• To address the lack of academic consolidation and standardization in ML applications for financial time series prediction.
• To identify and rank the most effective algorithm classes based on empirical performance across multiple markets and asset classes.
• To develop a standardized syntax for representing machine learning algorithms in text for improved reproducibility and comparison.
• To test the hypothesis that recurrent networks outperform feedforward models due to their ability to model long-term temporal dependencies.

Proposed method

• Conducted a systematic literature review of 150+ studies on ML in financial forecasting, extracting key experimental parameters.
• Built a comparative table across seven main parameters: algorithm type, dataset, forecasting horizon, performance metric, market, asset class, and geography.
• Proposed a standardized, human-readable syntax for describing machine learning algorithms using abbreviations and hierarchical notation.
• Collected performance metrics (e.g., accuracy, AUC, RMSE) from included studies and conducted rank analysis across algorithm classes.
• Applied non-parametric statistical ranking methods to assess relative performance of algorithm classes while minimizing bias from individual study designs.
• Used cross-validation and generalization metrics to ensure model robustness and avoid overfitting in the evaluation process.

Experimental results

Research questions

• RQ1 Which machine learning algorithm classes consistently outperform traditional stochastic models in financial time series forecasting?
• RQ2 To what extent do recurrent neural networks (e.g., LSTM, GRU) outperform feedforward networks and support vector machines in predicting financial returns?
• RQ3 Are there consistent performance differences across different asset classes (equities, forex, commodities) and geographic markets?
• RQ4 How do performance rankings vary when controlling for dataset, forecasting horizon, and evaluation metric?
• RQ5 Can a standardized syntax for describing ML algorithms improve reproducibility and comparability in financial forecasting research?

Key findings

• Machine learning algorithms, particularly recurrent neural networks, consistently outperform traditional stochastic models in financial market forecasting due to their ability to model non-linear and temporal dependencies.
• Recurrent neural networks (LSTM, GRU) achieve significantly higher performance rankings than feedforward networks and support vector machines across multiple datasets and markets.
• The study identifies a statistically significant advantage of RNNs in capturing long-term temporal dynamics in financial time series, suggesting exploitable patterns in market data.
• Performance rankings remain robust across different asset classes (equities, forex, commodities) and geographies, indicating broad applicability of RNN-based models.
• The proposed standardized syntax for describing ML algorithms enables clearer communication and reproducibility in future research.
• Support vector machines and feedforward networks show moderate performance but are outperformed by RNNs, especially in longer-horizon forecasting tasks.