[Paper Review] FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead

Kang Chen, Tao Han+12 more|arXiv (Cornell University)|Apr 6, 2023

Meteorological Phenomena and Simulations136 citations

TL;DR

FengWu is a data-driven global medium-range weather forecast system leveraging multi-modal, multi-task deep learning with a cross-modal Transformer, uncertainty loss, and a replay buffer to extend skillful forecast beyond 10 days. It achieves 10.75 days lead with ACC(z500) > 0.6 and outperforms GraphCast on 80% of targets while maintaining low inference cost.

ABSTRACT

We present FengWu, an advanced data-driven global medium-range weather forecast system based on Artificial Intelligence (AI). Different from existing data-driven weather forecast methods, FengWu solves the medium-range forecast problem from a multi-modal and multi-task perspective. Specifically, a deep learning architecture equipped with model-specific encoder-decoders and cross-modal fusion Transformer is elaborately designed, which is learned under the supervision of an uncertainty loss to balance the optimization of different predictors in a region-adaptive manner. Besides this, a replay buffer mechanism is introduced to improve medium-range forecast performance. With 39-year data training based on the ERA5 reanalysis, FengWu is able to accurately reproduce the atmospheric dynamics and predict the future land and atmosphere states at 37 vertical levels on a 0.25° latitude-longitude resolution. Hindcasts of 6-hourly weather in 2018 based on ERA5 demonstrate that FengWu performs better than GraphCast in predicting 80\% of the 880 reported predictands, e.g., reducing the root mean square error (RMSE) of 10-day lead global z500 prediction from 733 to 651 $m^{2}/s^2$. In addition, the inference cost of each iteration is merely 600ms on NVIDIA Tesla A100 hardware. The results suggest that FengWu can significantly improve the forecast skill and extend the skillful global medium-range weather forecast out to 10.75 days lead (with ACC of z500 > 0.6) for the first time.

Motivation & Objective

• Treat weather forecasting as a multi-modal, multi-task learning problem by separating variables as distinct modalities.
• Develop a modal-customized encoder–decoder architecture with a cross-modal Transformer to fuse information across modalities.
• Use an uncertainty loss to automatically balance multi-task predictions across variables, levels, and locations.
• Introduce a replay buffer to improve long-lead forecast accuracy and training efficiency.

Proposed method

• Represent weather data as multiple modalities corresponding to atmosphere variables and pressure levels.
• Use modal-customized encoders to extract features per modality and a cross-modal Transformer to fuse them.
• Apply modal-customized decoders to predict future states for each modality.
• Adopt an uncertainty loss to automatically weight different tasks via homoscedastic uncertainty.
• Implement a replay buffer that stores prior predictions to mimic autoregressive inference and improve long-lead forecasts.

Experimental results

Research questions

• RQ1 Can treating global medium-range weather forecasting as a multi-modal, multi-task problem improve forecast skill and lead time?
• RQ2 Does a cross-modal Transformer with modality-specific encoders/decoders outperform single-modal baselines and prior AI models like GraphCast?
• RQ3 Can uncertainty-based loss weighting automatically balance predictions across variables and pressure levels?
• RQ4 Does a replay buffer effectively enhance long-lead forecast accuracy and training efficiency?

Key findings

• FengWu extends skillful global medium-range forecast lead to 10.75 days for z500 (ACC>0.6) and 11.5 days for t2m.
• FengWu achieves higher accuracy than GraphCast on 80% of the 880 reported predictands.
• FengWu maintains low inference cost at 600 ms per iteration on NVIDIA Tesla A100 hardware.
• Compared to IFS, FengWu’s 10-day forecast for a single member is approximately 2000 times lower in energy consumption.
• The replay buffer is crucial for sustaining performance gains at longer lead times.
• Training used ERA5 data over 39 years at 0.25° resolution with 37 vertical levels and 189 predictands.