[Paper Review] Calibrating the CAMS European multi-model air quality forecasts for regional air pollution monitoring

Gabriele Casciaro, Mattia Cavaiola+1 more|arXiv (Cornell University)|Jan 31, 2022

Air Quality Monitoring and Forecasting31 references29 citations

TL;DR

This study develops a dynamic calibration strategy (D-EMOS+4r) using Ensemble Model Output Statistics (EMOS) to improve CAMS multi-model air quality forecasts for PM10, PM2.5, O3, NO2, and CO in Liguria, Italy. By integrating real-time observations, the calibrated forecasts significantly outperform both raw CAMS and higher-resolution models, while also providing reliable uncertainty estimates, enhancing early warning systems for regional air pollution.

ABSTRACT

The CAMS air quality multi-model forecasts have been assessed and calibrated for PM10, PM2.5, O3, NO2, and CO against observations collected by the Regional Monitoring Network of the Liguria region (northwestern Italy) in the years 2019 and 2020. The calibration strategy used in the present work has its roots in the well-established Ensemble Model Output Statistics (EMOS) through which a raw ensemble forecast can be accurately transformed into a predictive probability density function, with a simultaneous correction of biases and dispersion errors. The strategy also provides a calibrated forecast of model uncertainties. As a result of our analysis, the key role of pollutant real-time observations to be ingested in the calibration strategy clearly emerge especially in the shorter look-ahead forecast hours. Our dynamic calibration strategy turns out to be superior with respect to its analogous where real-time data are not taken into account. The best calibration strategy we have identified makes the CAMS multi-model forecast system more reliable than other raw air quality models running at higher spatial resolution which exploit more detailed information from inventory emission. We expect positive impacts of our research for identifying and set up reliable and economic air pollution early warning systems.

Motivation & Objective

• To improve the accuracy and reliability of CAMS multi-model air quality forecasts for regional air pollution monitoring in Liguria, Italy.
• To assess whether dynamic calibration using real-time observations enhances forecast performance compared to static or non-observing strategies.
• To evaluate the added value of CAMS multi-model forecasts against higher-resolution, higher-emission-inventory models.
• To develop a calibrated system that provides not only improved point forecasts but also reliable probabilistic uncertainty estimates.
• To support the design of cost-effective, reliable air pollution early warning systems through statistical post-processing.

Proposed method

• Application of a dynamic calibration strategy, D-EMOS+4r, based on Ensemble Model Output Statistics (EMOS) to transform raw CAMS ensemble forecasts into calibrated predictive probability density functions.
• Use of past observational data from the Liguria Regional Monitoring Network (2019–2020) to train the calibration model, with real-time observations ingested at each forecast step.
• Incorporation of time-varying bias correction and dispersion error adjustment via EMOS, ensuring both reliability and sharpness of forecasts.
• Use of the Probability Integral Transform (PIT) and Verification Rank (VR) histograms to assess forecast calibration and reliability.
• Employment of symmetrized skill scores (SS) and standard error indices (NMAE, NBI, HH, C, ∆) to quantitatively compare raw and calibrated forecasts against persistence and higher-resolution models.
• Evaluation of individual CAMS model members and ensemble median/mean to benchmark the performance of the calibrated system.

Experimental results

Research questions

• RQ1 Can a dynamic calibration strategy significantly improve the reliability and accuracy of CAMS multi-model air quality forecasts in a complex orographic region like Liguria?
• RQ2 Does the integration of real-time observational data during the calibration process lead to measurable performance gains, especially in short-lead-time forecasts?
• RQ3 How does the calibrated CAMS system compare in skill to raw, higher-resolution models that use more detailed emission inventories?
• RQ4 Can the calibration method simultaneously correct biases and provide reliable probabilistic forecasts of model uncertainty?
• RQ5 To what extent does dynamic calibration enhance the utility of CAMS forecasts for operational air pollution early warning systems?

Key findings

• The D-EMOS+4r dynamic calibration strategy significantly outperforms raw CAMS forecasts and persistence, especially in the first 24 forecast hours.
• The calibrated CAMS forecasts achieve higher skill than raw higher-resolution models that rely on more detailed emission inventories, demonstrating the intrinsic strength of the multi-model ensemble approach when properly calibrated.
• Real-time observations play a critical role in the calibration process, delivering clear added value—particularly in shorter-lead-time forecasts—by improving bias correction and uncertainty estimation.
• The calibrated system provides well-calibrated probabilistic forecasts, with PIT histograms indicating uniformity and a reliability index ∆ close to zero, confirming forecast calibration.
• The symmetrized skill score (SS) analysis confirms that the calibrated CAMS system achieves SS > 1 in key metrics, indicating superior performance over both persistence and raw higher-resolution models.
• The study establishes a new operational air quality prediction system for Liguria, demonstrating the feasibility of cost-effective, high-reliability early warning systems through statistical post-processing.