[Paper Review] Simple Baseline for Weather Forecasting Using Spatiotemporal Context Aggregation Network

Minseok Seo, Doyi Kim+4 more|arXiv (Cornell University)|Dec 6, 2022

Flood Risk Assessment and Management4 citations

TL;DR

This paper proposes SIANet, a simple yet effective spatiotemporal context aggregation network for short-term weather forecasting using only satellite images and 3D-CNNs, without multi-model ensembles or fine-tuning. It achieves state-of-the-art performance on 4 out of 5 W4C'22 benchmarks by leveraging kernel decomposition and a novel spatiotemporal refinement module, demonstrating that simplicity can match complexity in deep learning weather forecasting.

ABSTRACT

Traditional weather forecasting relies on domain expertise and computationally intensive numerical simulation systems. Recently, with the development of a data-driven approach, weather forecasting based on deep learning has been receiving attention. Deep learning-based weather forecasting has made stunning progress, from various backbone studies using CNN, RNN, and Transformer to training strategies using weather observations datasets with auxiliary inputs. All of this progress has contributed to the field of weather forecasting; however, many elements and complex structures of deep learning models prevent us from reaching physical interpretations. This paper proposes a SImple baseline with a spatiotemporal context Aggregation Network (SIANet) that achieved state-of-the-art in 4 parts of 5 benchmarks of W4C22. This simple but efficient structure uses only satellite images and CNNs in an end-to-end fashion without using a multi-model ensemble or fine-tuning. This simplicity of SIANet can be used as a solid baseline that can be easily applied in weather forecasting using deep learning.

Motivation & Objective

• To develop a simple, efficient deep learning baseline for short-term weather forecasting that avoids complex training pipelines.
• To reduce reliance on multi-model ensembles, fine-tuning, and auxiliary data like NWP or geographic coordinates.
• To improve spatiotemporal modeling in weather prediction using only 3D-CNNs and end-to-end training.
• To demonstrate that a minimal architecture with kernel decomposition and context aggregation can achieve state-of-the-art results.

Proposed method

• SIANet uses only satellite images (11 spectral bands) as input, with no additional static data such as latitude, longitude, or topography.
• The model is built entirely from 3D-convolutional layers, with kernel decomposition to reduce FLOPs by separating 3D filters into 2D spatial and 1D temporal components.
• A large spatial context aggregation module is introduced as the basic block, enhancing long-range spatial and temporal dependency learning.
• A spatiotemporal refinement module inspired by Markov chains is used to refine predictions across time steps in an end-to-end fashion.
• The model is trained end-to-end using binary cross-entropy loss on radar-reflectivity-derived rain rate maps.
• Ablation studies validate the contribution of each component: thresholding, center cropping, spatiotemporal module, and large context aggregation module.

Experimental results

Research questions

• RQ1 Can a simple 3D-CNN-based model achieve state-of-the-art performance in weather forecasting without multi-model ensembles or fine-tuning?
• RQ2 How effective is kernel decomposition in reducing FLOPs while preserving performance in weather forecasting with 3D-CNNs?
• RQ3 To what extent can a large spatial context aggregation module improve spatiotemporal modeling in short-term precipitation prediction?
• RQ4 Does an end-to-end spatiotemporal refinement module enhance prediction accuracy compared to standard regression heads?
• RQ5 Can a model trained only on satellite data outperform more complex models relying on auxiliary data like NWP outputs?

Key findings

• SIANet achieved state-of-the-art performance on 4 out of 5 benchmarks in the W4C'22 challenge, with a mIoU of 0.465 on the best configuration.
• The ablation study showed that adding components like CenterCrop, STM, and LCAM progressively improved mIoU by up to 12.7 percentage points over the baseline.
• The model demonstrated dynamic prediction behavior across time steps, generating distinct outputs for each of the 8-hour forecast horizon rather than a single average prediction.
• The model successfully captured cloud movement patterns, as shown in qualitative results where predictions evolved with the direction of storm systems.
• Despite strong performance, the model struggled with rain rate classification, overpredicting light-to-moderate rainfall (threshold 0.2 mm/h), indicating a need for multi-class or regression loss in future work.
• The use of kernel decomposition significantly reduced FLOPs while maintaining high performance, proving effective for large-scale weather data training.