[论文解读] AI Foundation Models for Weather and Climate: Applications, Design, and Implementation

S. Karthik Mukkavilli, Daniel Civitarese+13 more|arXiv (Cornell University)|Sep 19, 2023

Meteorological Phenomena and Simulations被引用 14

一句话总结

本文综述了AI基础模型在天气与气候中的兴起，概述应用、设计选择和实现考量，朝着通用的天气基础模型发展。

ABSTRACT

Machine learning and deep learning methods have been widely explored in understanding the chaotic behavior of the atmosphere and furthering weather forecasting. There has been increasing interest from technology companies, government institutions, and meteorological agencies in building digital twins of the Earth. Recent approaches using transformers, physics-informed machine learning, and graph neural networks have demonstrated state-of-the-art performance on relatively narrow spatiotemporal scales and specific tasks. With the recent success of generative artificial intelligence (AI) using pre-trained transformers for language modeling and vision with prompt engineering and fine-tuning, we are now moving towards generalizable AI. In particular, we are witnessing the rise of AI foundation models that can perform competitively on multiple domain-specific downstream tasks. Despite this progress, we are still in the nascent stages of a generalizable AI model for global Earth system models, regional climate models, and mesoscale weather models. Here, we review current state-of-the-art AI approaches, primarily from transformer and operator learning literature in the context of meteorology. We provide our perspective on criteria for success towards a family of foundation models for nowcasting and forecasting weather and climate predictions. We also discuss how such models can perform competitively on downstream tasks such as downscaling (super-resolution), identifying conditions conducive to the occurrence of wildfires, and predicting consequential meteorological phenomena across various spatiotemporal scales such as hurricanes and atmospheric rivers. In particular, we examine current AI methodologies and contend they have matured enough to design and implement a weather foundation model.

研究动机与目标

• 推动全球与区域天气和气候问题的基础模型发展。
• 总结气象学领域的最先进AI方法（ transformers、图神经网络、物理信息ML）等。
• 讨论天气基础模型的设计标准、数据需求和评估考量。
• 识别基础模型在下游任务中可能出众的领域（nowcasting 近时预报、下采样/尺度化、参量化、数据同化）。
• 勾勒天气基础模型家族的路线图与成功标准。

提出的方法

• 回顾气象领域当前的最前沿AI方法（transformers、图神经网络、算子学习）。
• 提出用于下游任务的编码-解码基础模型结构及微调范式的概念化。
• 讨论天气/气候情境下自监督学习的预训练数据需求及损失函数考量。
• 分析骨干网络（transformers 与基于图的网络）及多尺度数据表示之间的设计权衡。
• 提出与气象真实感和下游应用效用相一致的评估与诊断标准。

实验结果

研究问题

• RQ1 气象与气候基础模型能够有效应用的关键下游任务有哪些？
• RQ2 构建可泛化的天气基础模型所需的设计准则与数据需求是什么？
• RQ3 基础模型如何提升下采样/尺度化、nowcasting、数据同化和参数化等任务？
• RQ4 将FM方法应用于天气/气候领域面临的挑战与局限性有哪些，如何减轻？
• RQ5 长期部署天气基础模型所需的路线图与里程碑有哪些？

主要发现

• 基础模型有潜力在天气/气候变量与时间尺度上提升准确性和效率。
• 在大型、多样化数据集上的预训练减少了下游任务微调对带标签数据的需求。
• 使用轻量级任务特定解码器进行微调实现多任务的灵活部署。
• 当前的AI仿真器和模型在与部分数值天气预测(NWP)系统的性能方面可比，但在网格粒度、提前时间和泛化方面存在局限。
• 下采样、灾害检测和气候应用可以通过更好的泛化性和数据效率从FM方法中受益。
• 著名示例（如 FourCastNet、PanguWeather、GraphCast）展示了向可扩展、高分辨率建模的快速进展。