[Paper Review] Pattern-based Long Short-term Memory for Mid-term Electrical Load Forecasting

Paweł Pełka, Grzegorz Dudek|arXiv (Cornell University)|Apr 22, 2020

Energy Load and Power Forecasting27 references4 citations

TL;DR

This paper proposes a pattern-based Long Short-Term Memory (LSTM) model for mid-term electrical load forecasting, using x-patterns to simplify nonlinear, nonstationary time series by removing trends and stabilizing variance. The hybrid LSTMx model forecasts patterns via sequence-to-sequence LSTM and coding variables via exponential smoothing, achieving superior accuracy over standard LSTM and classical models across 35 European countries over a 12-month horizon.

ABSTRACT

This work presents a Long Short-Term Memory (LSTM) network for forecasting a monthly electricity demand time series with a one-year horizon. The novelty of this work is the use of pattern representation of the seasonal time series as an alternative to decomposition. Pattern representation simplifies the complex nonlinear and nonstationary time series, filtering out the trend and equalizing variance. Two types of patterns are defined: x-pattern and y-pattern. The former requires additional forecasting for the coding variables. The latter determines the coding variables from the process history. A hybrid approach based on x-patterns turned out to be more accurate than the standard LSTM approach based on a raw time series. In this combined approach an x-pattern is forecasted using a sequence-to-sequence LSTM network and the coding variables are forecasted using exponential smoothing. A simulation study performed on the monthly electricity demand time series for 35 European countries confirmed the high performance of the proposed model and its competitiveness to classical models such as ARIMA and exponential smoothing as well as the MLP neural network model.

Motivation & Objective

• To improve mid-term electrical load forecasting accuracy by simplifying complex, nonlinear, and nonstationary time series.
• To address the limitations of raw time series in LSTM models through pattern-based preprocessing that removes trends and equalizes variance.
• To develop and evaluate two pattern types—x-pattern and y-pattern—where x-pattern requires separate forecasting of coding variables.
• To compare the performance of the proposed pattern-based LSTM models against standard LSTM, ARIMA, exponential smoothing, and MLP models.
• To assess the impact of model ensembling on forecast robustness and error reduction.

Proposed method

• The method uses x-patterns, which represent normalized seasonal subsequences of the original time series, to simplify forecasting by filtering out trends and stabilizing variance.
• For the x-pattern approach, a sequence-to-sequence LSTM network forecasts the pattern sequence, while exponential smoothing predicts the coding variables (e.g., level and trend) for reconstruction.
• The y-pattern approach determines coding variables from the previous year’s pattern, introducing a temporal dependency that complicates learning.
• The final forecast is reconstructed by reapplying the current trend and variance to the predicted pattern, restoring the original scale.
• Model ensembling is implemented by averaging forecasts from 100 independent training runs to improve robustness and reduce error.
• The model is evaluated on monthly electricity demand data from 35 European countries over a 12-month forecasting horizon.

Experimental results

Research questions

• RQ1 Can pattern-based preprocessing improve the forecasting accuracy of LSTM models for mid-term electrical load forecasting?
• RQ2 How does the x-pattern approach, which separates pattern and coding variable forecasting, compare to the y-pattern approach in terms of accuracy and stability?
• RQ3 Does the hybrid LSTMx model, combining sequence-to-sequence LSTM with exponential smoothing, outperform standard LSTM and classical statistical models?
• RQ4 To what extent does model ensembling reduce forecast error and improve robustness in the proposed framework?
• RQ5 How do the proposed models perform relative to ARIMA, exponential smoothing, and MLP models on real-world European electricity demand data?

Key findings

• The LSTMx model, which uses x-patterns and a hybrid forecasting approach, achieved the lowest MAPE (4.99%) among all LSTM variants, outperforming standard LSTM and LSTMy.
• The LSTMx ensemble model reduced the median APE to 3.01% and MAPE to 4.99%, demonstrating significant error reduction through averaging over 100 independent runs.
• For country-specific forecasts, LSTMx achieved a MAPE of 1.79% in Poland and 1.60% in Germany when using ensembles, outperforming other models.
• The LSTMy model performed worse than LSTMx due to delayed coding variable estimation, which disrupted the time series and increased forecast error.
• The proposed pattern-based approach significantly improved LSTM performance by simplifying the underlying time series into more regular, normalized seasonal patterns.
• The results show that the LSTMx model is competitive with classical models such as exponential smoothing and MLP, particularly in the second half of the year where improvements were most pronounced.