[论文解读] Foundation Models for Weather and Climate Data Understanding: A Comprehensive Survey

Shengchao Chen, Guodong Long+3 more|arXiv (Cornell University)|Dec 5, 2023

Meteorological Phenomena and Simulations被引用 10

一句话总结

本文综述了天气与气候数据的前沿基础模型与任务特定模型，聚焦时间序列、时空数据和文本数据，讨论数据集、架构、应用、挑战与未来方向。

ABSTRACT

As artificial intelligence (AI) continues to rapidly evolve, the realm of Earth and atmospheric sciences is increasingly adopting data-driven models, powered by progressive developments in deep learning (DL). Specifically, DL techniques are extensively utilized to decode the chaotic and nonlinear aspects of Earth systems, and to address climate challenges via understanding weather and climate data. Cutting-edge performance on specific tasks within narrower spatio-temporal scales has been achieved recently through DL. The rise of large models, specifically large language models (LLMs), has enabled fine-tuning processes that yield remarkable outcomes across various downstream tasks, thereby propelling the advancement of general AI. However, we are still navigating the initial stages of crafting general AI for weather and climate. In this survey, we offer an exhaustive, timely overview of state-of-the-art AI methodologies specifically engineered for weather and climate data, with a special focus on time series and text data. Our primary coverage encompasses four critical aspects: types of weather and climate data, principal model architectures, model scopes and applications, and datasets for weather and climate. Furthermore, in relation to the creation and application of foundation models for weather and climate data understanding, we delve into the field's prevailing challenges, offer crucial insights, and propose detailed avenues for future research. This comprehensive approach equips practitioners with the requisite knowledge to make substantial progress in this domain. Our survey encapsulates the most recent breakthroughs in research on large, data-driven models for weather and climate data understanding, emphasizing robust foundations, current advancements, practical applications, crucial resources, and prospective research opportunities.

研究动机与目标

• 提供关于天气与气候数据的大型基础模型与任务特定模型的最新概览。
• 按架构和数据模态（时间序列、时空数据、文本）对模型进行分类。
• 概述可用于天气/气候 AI 研究的数据集、工具与资源。
• 突出气候基础模型的挑战、机遇与设计考量。

提出的方法

• 对应用于天气与气候数据的基础模型和任务特定模型进行系统文献综述。
• 按架构（RNNs、Transformers、GANs、Diffusion、GNNs）和数据模态（时间序列、时空、文本）对模型进行分类。
• 讨论数据类型、任务以及天气/气候科学中的典型应用。
• 汇编与该领域相关的数据集、资源和开源工具。
• 阐述气候基础模型的设计原则与未来研究方向。

实验结果

研究问题

• RQ1 当前用于天气与气候数据理解的基础模型和任务特定模型有哪些？
• RQ2 这些模型如何在数据模态和体系结构家族中分布？
• RQ3 有哪些数据集与工具可用于支持天气与气候 AI 的研究？
• RQ4 构建天气/气候基础模型面临哪些挑战与机遇？

主要发现

• 本综述提供了关于天气与气候数据在时间序列、时空数据和文本方面的大规模模型与任务特定模型的全面、当代概览。
• 它提供了对气候基础模型和任务特定模型的结构化分类，按 RNNs、Transformers、GANs、Diffusion 模型和 GNNs 等架构进行详细区分。
• 提供了丰富的数据集、开源实现和资源的汇编，以帮助研究人员。
• 本文识别挑战并概述在数据表示、多模态建模、可解释性、泛化、隐私和持续学习方面的未来研究方向。
• 它为构建天气与气候基础模型提供设计洞见，包括数据选择、表达、学习策略和评估方案。