[Paper Review] Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark

Jesus Lago, Grzegorz Marcjasz+2 more|arXiv (Cornell University)|Aug 18, 2020

Energy Load and Power Forecasting152 references406 citations

TL;DR

This paper proposes a rigorous benchmark for day-ahead electricity price forecasting by introducing open-access datasets from five major markets, open-source implementations of state-of-the-art models (LEAR and DNN), and a comprehensive set of best practices. It demonstrates that deep learning ensembles outperform statistical models in accuracy, while LEAR remains optimal for low-latency applications due to its computational efficiency and strong performance.

ABSTRACT

While the field of electricity price forecasting has benefited from plenty of contributions in the last two decades, it arguably lacks a rigorous approach to evaluating new predictive algorithms. The latter are often compared using unique, not publicly available datasets and across too short and limited to one market test samples. The proposed new methods are rarely benchmarked against well established and well performing simpler models, the accuracy metrics are sometimes inadequate and testing the significance of differences in predictive performance is seldom conducted. Consequently, it is not clear which methods perform well nor what are the best practices when forecasting electricity prices. In this paper, we tackle these issues by performing a literature survey of state-of-the-art models, comparing state-of-the-art statistical and deep learning methods across multiple years and markets, and by putting forward a set of best practices. In addition, we make available the considered datasets, forecasts of the state-of-the-art models, and a specifically designed python toolbox, so that new algorithms can be rigorously evaluated in future studies.

Motivation & Objective

• Address the lack of standardized, reproducible evaluation in electricity price forecasting (EPF) research.
• Provide long-term, open-access benchmark datasets spanning six years across five major electricity markets.
• Develop and release open-source implementations of state-of-the-art models (LEAR and DNN) for fair comparison.
• Establish a set of best practices to improve methodological rigor, reproducibility, and statistical validity in EPF studies.
• Enable future research to evaluate new models under consistent conditions using shared datasets and benchmark models.

Proposed method

• Curated five open-access, long-term (6-year) datasets of day-ahead electricity prices from major European and North American markets, each including hourly prices and two exogenous variables.
• Implemented and open-sourced two state-of-the-art models: the Lasso Estimated AutoRegressive (LEAR) model for statistical forecasting and a Deep Neural Network (DNN) with Bayesian hyperparameter and feature optimization.
• Developed a dedicated Python library, epftoolbox, to host datasets, model forecasts, and evaluation tools for consistent benchmarking.
• Applied standardized evaluation metrics including rMAE, sMAPE, MAPE, and RMSE, with statistical testing to assess significance of performance differences.
• Used a strict out-of-sample evaluation protocol with daily recalibration, separate training/validation/test splits, and explicit data partitioning to ensure reproducibility.
• Evaluated model performance across all five datasets using both accuracy metrics and non-parametric statistical tests to validate significance of differences.

Experimental results

Research questions

• RQ1 Which forecasting models consistently outperform others across diverse, long-term electricity markets?
• RQ2 How do deep learning models compare to well-established statistical models like LEAR in terms of accuracy and computational cost?
• RQ3 What are the most reliable and statistically sound evaluation metrics for electricity price forecasting?
• RQ4 To what extent do ensemble deep learning models improve upon individual DNNs in forecasting accuracy?
• RQ5 What methodological best practices ensure reproducibility and meaningful comparisons in future EPF research?

Key findings

• Deep neural network (DNN) ensembles significantly outperform all other models in forecasting accuracy, with statistically significant improvements over individual models and the LEAR model.
• The LEAR model achieves performance very close to that of DNNs but with computational costs up to 100 times lower, making it ideal for real-time decision-making applications.
• The rMAE metric is more reliable than MAPE and sMAPE, as MAPE is unreliable due to its sensitivity to near-zero prices and can lead to misleading conclusions.
• Statistical testing is essential to validate whether performance differences between models are significant; without it, conclusions from EPF studies are often invalid.
• Test datasets must span at least one year to ensure representative evaluation; shorter test periods, such as one-week samples, lead to unreliable and non-generalizable results.
• The open-access benchmark, including datasets and pre-computed forecasts of state-of-the-art models, enables fair, reproducible, and consistent evaluation of new forecasting methods.