[Paper Review] Electricity Price Forecasting in the Irish Balancing Market

Ciaran O’Connor, Joseph Collins+2 more|arXiv (Cornell University)|Feb 9, 2024

Energy Efficiency and ManagementEnergy3 citations

TL;DR

This paper proposes a benchmark framework for electricity price forecasting in the Irish Balancing Market (BM), evaluating statistical, machine learning, and deep learning models trained on a newly released public dataset. Despite strong performance in the less volatile Day-Ahead Market, deep learning models underperform in the BM due to its extreme volatility; instead, the LASSO-based LEAR model achieves the best accuracy, outperforming complex DL approaches, highlighting the need for simpler, more robust models in highly volatile markets.

ABSTRACT

Short-term electricity markets are becoming more relevant due to less-predictable renewable energy sources, attracting considerable attention from the industry. The balancing market is the closest to real-time and the most volatile among them. Its price forecasting literature is limited, inconsistent and outdated, with few deep learning attempts and no public dataset. This work applies to the Irish balancing market a variety of price prediction techniques proven successful in the widely studied day-ahead market. We compare statistical, machine learning, and deep learning models using a framework that investigates the impact of different training sizes. The framework defines hyperparameters and calibration settings; the dataset and models are made public to ensure reproducibility and to be used as benchmarks for future works. An extensive numerical study shows that well-performing models in the day-ahead market do not perform well in the balancing one, highlighting that these markets are fundamentally different constructs. The best model is LEAR, a statistical approach based on LASSO, which outperforms more complex and computationally demanding approaches.

Motivation & Objective

• To establish a reproducible, open-source framework for forecasting electricity prices in the Irish Balancing Market (BM), a highly volatile and under-researched market.
• To evaluate the performance of state-of-the-art models—statistical, machine learning, and deep learning—originally developed for the more stable Day-Ahead Market (DAM) in the context of the BM.
• To investigate the impact of training data size on model accuracy across different model classes in the BM setting.
• To identify fundamental differences between the DAM and BM that affect model performance, particularly the challenges faced by deep learning in handling extreme price spikes.
• To provide a public dataset and codebase to serve as a benchmark for future research in BM price forecasting.

Proposed method

• The study uses a public dataset of hourly Irish Balancing Market prices and associated explanatory variables (e.g., load, renewables, DAM prices) collected from 2017 to 2020.
• A standardized training and evaluation framework is implemented, with consistent hyperparameter tuning and calibration procedures across all models to ensure fair comparison.
• The framework evaluates 10 models: statistical (LEAR, SVR), tree-based (Random Forest, XGBoost), and deep learning (SH DNN, MH DNN), with training sizes varying from 1 to 365 days.
• Model performance is measured using Mean Absolute Error (MAE), with results reported across multiple test splits to ensure robustness.
• Computational efficiency is assessed by measuring training time on CPU (for non-DL models) and GPU (for DL models), reflecting real-time deployment constraints.
• The analysis includes visual comparisons of predictions against actual prices to diagnose overfitting and misprediction of spikes.

Figure 1 : Schematic overview of the typical sequence of existing electricity markets in the EU. Markets in dotted lines are optional Meeus [ 2020 ]

Experimental results

Research questions

• RQ1 How do models that perform well in the Day-Ahead Market (DAM) perform when applied to the more volatile Irish Balancing Market (BM)?
• RQ2 What is the impact of varying training data size on forecast accuracy across statistical, machine learning, and deep learning models in the BM context?
• RQ3 Why do deep learning models underperform in the BM despite their success in the DAM, and what structural or data-related factors contribute to this gap?
• RQ4 Which model class—statistical, machine learning, or deep learning—achieves the best trade-off between accuracy and computational efficiency in the BM?
• RQ5 Can simpler models like LEAR, based on LASSO, outperform complex deep neural networks in highly volatile electricity markets?

Key findings

• The LEAR model, a LASSO-based statistical approach, achieves the lowest Mean Absolute Error (MAE) of €4.21/MWh, outperforming more complex models including deep learning architectures.
• Deep learning models such as SH DNN and MH DNN exhibit significantly higher errors in the BM—up to €2.81/MWh higher than LEAR—due to overfitting to recent price spikes and poor generalization to volatile fluctuations.
• The performance gap between models is more pronounced in the BM than in the DAM, indicating that market structure and data volatility fundamentally alter model effectiveness.
• Training data size has a notable impact: models trained on full-year data (365 days) achieve better accuracy than those trained on shorter sequences, though the improvement diminishes beyond a certain point.
• Computational efficiency favors simpler models: LEAR and XGBoost require only 1–3 minutes to train, while deep learning models take up to 30 minutes, with MH DNN being the slowest due to high parameter count.
• The SH DNN model performs well in the DAM, with a MAE of €7.02/MWh, but its performance degrades significantly in the BM, underscoring the limitations of DL in handling extreme volatility.

Figure 2 : MH RNN DNN Model