[Paper Review] Open-Source Ground-based Sky Image Datasets for Very Short-term Solar Forecasting, Cloud Analysis and Modeling: A Comprehensive Survey

Yuhao Nie, Xiatong Li+4 more|arXiv (Cornell University)|Nov 27, 2022

Solar Radiation and Photovoltaics4 citations

TL;DR

This paper presents a comprehensive survey of 72 open-source ground-based sky image datasets for very short-term solar forecasting, cloud analysis, and modeling. It introduces a multi-criteria ranking system across eight dimensions to evaluate dataset suitability, enabling researchers to select optimal data for deep learning applications in solar irradiance prediction, cloud segmentation, classification, and motion estimation.

ABSTRACT

Sky-image-based solar forecasting using deep learning has been recognized as a promising approach in reducing the uncertainty in solar power generation. However, one of the biggest challenges is the lack of massive and diversified sky image samples. In this study, we present a comprehensive survey of open-source ground-based sky image datasets for very short-term solar forecasting (i.e., forecasting horizon less than 30 minutes), as well as related research areas which can potentially help improve solar forecasting methods, including cloud segmentation, cloud classification and cloud motion prediction. We first identify 72 open-source sky image datasets that satisfy the needs of machine/deep learning. Then a database of information about various aspects of the identified datasets is constructed. To evaluate each surveyed datasets, we further develop a multi-criteria ranking system based on 8 dimensions of the datasets which could have important impacts on usage of the data. Finally, we provide insights on the usage of these datasets for different applications. We hope this paper can provide an overview for researchers who are looking for datasets for very short-term solar forecasting and related areas.

Motivation & Objective

• To identify and catalog open-source ground-based sky image datasets suitable for machine and deep learning applications in solar forecasting.
• To address the critical challenge of data scarcity and diversity in very short-term solar forecasting by compiling a global, comprehensive dataset inventory.
• To develop a multi-criteria evaluation framework to rank datasets based on factors impacting model performance and usability.
• To provide researchers with actionable insights and access pathways for selecting appropriate datasets for cloud segmentation, classification, motion prediction, and solar irradiance forecasting.

Proposed method

• Systematically surveyed and collected 72 open-source sky image datasets from diverse geographic and climatic regions, focusing on ground-based all-sky imagers (ASIs) and sky-polarimetric images (SPIs).
• Constructed a detailed database containing metadata across eight dimensions: geographic location, temporal and spatial resolution, image format, cloud labeling type, number of images, image size, temporal coverage, and data access method.
• Designed a multi-criteria ranking system using eight evaluation dimensions—such as image resolution, temporal frequency, cloud annotation quality, and data accessibility—to rank datasets for different application needs.
• Evaluated each dataset based on its suitability for specific tasks including very short-term solar forecasting, cloud segmentation, cloud classification, and cloud motion prediction.
• Provided direct access links and usage guidelines for each dataset to facilitate adoption in deep learning and computer vision research.
• Categorized datasets by primary application focus (e.g., solar forecasting, cloud analysis) and cross-referenced with published studies to validate real-world usage.

Experimental results

Research questions

• RQ1 Which open-source ground-based sky image datasets are most suitable for very short-term solar forecasting (forecast horizon <30 min)?
• RQ2 How do different dataset characteristics—such as image resolution, temporal frequency, and cloud annotation quality—affect performance in deep learning models for solar irradiance prediction?
• RQ3 What are the most commonly used datasets in recent research on cloud segmentation, classification, and motion prediction using sky images?
• RQ4 How do the geographic and climatic distributions of available datasets influence the generalization capability of solar forecasting models?
• RQ5 What criteria should researchers prioritize when selecting a sky image dataset for a specific application in solar energy forecasting or cloud modeling?

Key findings

• A total of 72 open-source ground-based sky image datasets were identified and cataloged, covering diverse climate zones and geographic locations worldwide.
• The dataset database includes comprehensive metadata across eight dimensions, enabling systematic comparison and selection of datasets for specific research needs.
• The multi-criteria ranking system successfully identifies top-performing datasets for each application area, with datasets like SRRL-BMS, SIRTA, ARM-SGP, and SKIPP’D emerging as highly ranked for solar forecasting.
• Cloud classification datasets such as MGCD, GRSCD, and WMD were found to have strong annotation quality with WMO-based cloud type labels at image or pixel level.
• The survey revealed that 19 datasets were used in 140+ published studies, confirming their active role in advancing solar forecasting and cloud modeling research.
• Despite the abundance of datasets, significant gaps remain in coverage for tropical and arid climates, highlighting the need for more diverse data collection.