[Paper Review] KARINA: An Efficient Deep Learning Model for Global Weather Forecast

Minjong Cheon, Yo-Hwan Choi+4 more|arXiv (Cornell University)|Mar 13, 2024

Computational Physics and Python Applications4 citations

TL;DR

KARINA is a computationally efficient deep learning model for global weather forecasting that combines ConvNext, SENet, and Geocyclic Padding to achieve forecasting accuracy comparable to high-resolution models like Pangu-Weather and GraphCast—using only 4 NVIDIA A100 GPUs and under 12 hours of training. It outperforms ECMWF S2S reforecasts at 7-day lead times and matches SOTA models despite training on 2.5° resolution data with 100× fewer pixels.

ABSTRACT

Deep learning-based, data-driven models are gaining prevalence in climate research, particularly for global weather prediction. However, training the global weather data at high resolution requires massive computational resources. Therefore, we present a new model named KARINA to overcome the substantial computational demands typical of this field. This model achieves forecasting accuracy comparable to higher-resolution counterparts with significantly less computational resources, requiring only 4 NVIDIA A100 GPUs and less than 12 hours of training. KARINA combines ConvNext, SENet, and Geocyclic Padding to enhance weather forecasting at a 2.5° resolution, which could filter out high-frequency noise. Geocyclic Padding preserves pixels at the lateral boundary of the input image, thereby maintaining atmospheric flow continuity in the spherical Earth. SENet dynamically improves feature response, advancing atmospheric process modeling, particularly in the vertical column process as numerous channels. In this vein, KARINA sets new benchmarks in weather forecasting accuracy, surpassing existing models like the ECMWF S2S reforecasts at a lead time of up to 7 days. Remarkably, KARINA achieved competitive performance even when compared to the recently developed models (Pangu-Weather, GraphCast, ClimaX, and FourCastNet) trained with high-resolution data having 100 times larger pixels. Conclusively, KARINA significantly advances global weather forecasting by efficiently modeling Earth's atmosphere with improved accuracy and resource efficiency.

Motivation & Objective

• To address the high computational cost of training high-resolution deep learning models for global weather forecasting.
• To improve forecasting accuracy at 2.5° resolution without relying on massive computational resources.
• To enhance spatial and vertical feature representation in climate data using novel architectural components.
• To develop a model that achieves SOTA performance with significantly reduced training time and hardware requirements.

Proposed method

• KARINA integrates ConvNext for efficient processing of high-dimensional spatial data in the ERA5 dataset.
• It incorporates SENet for channel-wise feature recalibration, improving representation of atmospheric processes across vertical columns.
• Geocyclic Padding is introduced to preserve boundary continuity at the edges of spherical Earth projections, maintaining atmospheric flow integrity.
• The model is trained on daily-mean ERA5 reanalysis data at 2.5° resolution using a standard autoregressive forecasting setup.
• A fine-tuning strategy is applied to enhance performance on specific variables, improving generalization and robustness.
• The architecture is optimized for inference efficiency and scalability, enabling rapid training on minimal hardware.

Experimental results

Research questions

• RQ1 Can a deep learning model achieve SOTA weather forecasting accuracy at 2.5° resolution with minimal computational resources?
• RQ2 How does Geocyclic Padding improve model performance at the equatorial and lateral boundaries of global weather data?
• RQ3 To what extent does SENet enhance vertical atmospheric process modeling in a ConvNext-based architecture?
• RQ4 How does KARINA compare in skill to ECMWF S2S reforecasts and other SOTA models like Pangu-Weather and GraphCast?
• RQ5 Can a model trained on lower-resolution data match the performance of models trained on 100× higher-resolution inputs?

Key findings

• KARINA achieved a 7-day forecast RMSE of 463.65 for Z500, closely matching Pangu-Weather (504.90) and GraphCast (466.53), despite using only 1/100th the pixel count.
• The model outperformed ECMWF S2S reforecasts across all 7 forecast days for TCWV, T2M, and Z500, with lower RMSE throughout the lead time.
• Geocyclic Padding significantly improved the amplitude of high-pressure systems and cold anomalies in Northern Europe, closely matching ERA5’s North Atlantic Oscillation patterns.
• SENet enhanced the model’s ability to capture fine-scale features such as southwesterly winds and warming in Northern Africa.
• KARINA achieved competitive performance with less than 12 hours of training on just 4 NVIDIA A100 GPUs, a fraction of the resources required by models like Pangu-Weather (192 V100s, 64 days).
• The tuned version of KARINA (KARINA_Tuned) reduced RMSE by 0.8% on average compared to the base model, demonstrating the effectiveness of the fine-tuning strategy.