[Paper Review] Machine learning methods for postprocessing ensemble forecasts of wind gusts: A systematic comparison

Benedikt Schulz, Sebastian Lerch|arXiv (Cornell University)|Jun 17, 2021

Meteorological Phenomena and Simulations70 references106 citations

TL;DR

This paper proposes a systematic comparison of eight statistical and machine learning methods for postprocessing ensemble wind gust forecasts, using six years of high-resolution data from Germany's operational weather service. It demonstrates that neural network-based approaches—particularly a flexible, locally adaptive framework incorporating additional meteorological predictors—significantly outperform traditional methods by learning physically consistent relationships, especially related to the diurnal planetary boundary layer transition.

ABSTRACT

Postprocessing ensemble weather predictions to correct systematic errors has become a standard practice in research and operations. However, only few recent studies have focused on ensemble postprocessing of wind gust forecasts, despite its importance for severe weather warnings. Here, we provide a comprehensive review and systematic comparison of eight statistical and machine learning methods for probabilistic wind gust forecasting via ensemble postprocessing, that can be divided in three groups: State of the art postprocessing techniques from statistics (ensemble model output statistics (EMOS), member-by-member postprocessing, isotonic distributional regression), established machine learning methods (gradient-boosting extended EMOS, quantile regression forests) and neural network-based approaches (distributional regression network, Bernstein quantile network, histogram estimation network). The methods are systematically compared using six years of data from a high-resolution, convection-permitting ensemble prediction system that was run operationally at the German weather service, and hourly observations at 175 surface weather stations in Germany. While all postprocessing methods yield calibrated forecasts and are able to correct the systematic errors of the raw ensemble predictions, incorporating information from additional meteorological predictor variables beyond wind gusts leads to significant improvements in forecast skill. In particular, we propose a flexible framework of locally adaptive neural networks with different probabilistic forecast types as output, which not only significantly outperform all benchmark postprocessing methods but also learn physically consistent relations associated with the diurnal cycle, especially the evening transition of the planetary boundary layer.

Motivation & Objective

• To address the lack of comprehensive studies on ensemble postprocessing for wind gusts, a critical variable for severe weather warnings.
• To systematically compare a wide range of statistical and machine learning methods tailored specifically for probabilistic wind gust forecasting.
• To evaluate the impact of incorporating additional meteorological predictor variables beyond raw wind gust ensemble outputs.
• To investigate whether advanced machine learning models can learn physically consistent relationships, such as those tied to the diurnal cycle.
• To provide a flexible, modular framework for neural network-based postprocessing that supports multiple probabilistic forecast types.

Proposed method

• The study evaluates eight postprocessing methods: three statistical (EMOS, MBM, IDR), two established ML (EMOS-GB, QRF), and three neural network-based (DRN, BQN, HEN).
• All methods are trained on six years of hourly data from a 20-member convection-permitting ensemble prediction system (COSMO-DE-EPS) and 175 surface weather stations in Germany.
• The neural network-based framework (DRN, BQN, HEN) uses input features including wind gusts, temperature, humidity, and large-scale predictors, with outputs ranging from parametric distributions to quantile functions and histograms.
• Feature importance is assessed via permutation importance, with results visualized across lead times to assess temporal dynamics.
• The models are evaluated using proper scoring rules (e.g., CRPS) and reliability diagrams to assess calibration and sharpness.
• A modular, locally adaptive design allows for joint modeling across stations and lead times, enhancing forecast consistency.

Experimental results

Research questions

• RQ1 How do different machine learning and statistical postprocessing methods compare in terms of predictive performance for wind gusts?
• RQ2 To what extent does including additional meteorological predictors (beyond wind gusts) improve forecast skill?
• RQ3 Can neural network-based models learn physically consistent relationships, such as those associated with the diurnal planetary boundary layer transition?
• RQ4 How do feature importance patterns vary across lead times and models, and what do they reveal about the underlying meteorological processes?
• RQ5 Can a flexible, modular neural network framework outperform established benchmark methods while preserving physical plausibility?

Key findings

• All postprocessing methods significantly improve calibration and reduce systematic errors compared to raw ensemble forecasts.
• Incorporating additional meteorological predictors—such as temperature, humidity, and wind speed—leads to substantial improvements in forecast skill across all methods.
• Neural network-based models (DRN, BQN, HEN) consistently outperform all benchmark methods in terms of CRPS and reliability, with the HEN variant showing the strongest performance.
• The proposed modular, locally adaptive neural network framework learns physically consistent relationships, particularly capturing the evening transition of the planetary boundary layer.
• Permutation importance analysis reveals that wind gust ensemble spread and temperature-related variables are among the most influential predictors, with their importance varying systematically across lead times.
• The DRN, BQN, and HEN models demonstrate robustness and adaptability, with feature importance patterns aligning with known meteorological dynamics, such as the diurnal cycle and storm-related processes.