[Paper Review] EarthNets: Empowering AI in Earth Observation

Zhitong Xiong, Fahong Zhang+3 more|arXiv (Cornell University)|Oct 10, 2022

Scientific Computing and Data Management60 citations

TL;DR

The paper reviews over 400 publicly available Earth Observation datasets, analyzes their attributes to build a new benchmark, and releases EarthNets for fair, reproducible evaluation of deep learning models on remote sensing data.

ABSTRACT

Earth observation (EO), aiming at monitoring the state of planet Earth using remote sensing data, is critical for improving our daily lives and living environment. With a growing number of satellites in orbit, an increasing number of datasets with diverse sensors and research domains are being published to facilitate the research of the remote sensing community. This paper presents a comprehensive review of more than 500 publicly published datasets, including research domains like agriculture, land use and land cover, disaster monitoring, scene understanding, vision-language models, foundation models, climate change, and weather forecasting. We systematically analyze these EO datasets from four aspects: volume, resolution distributions, research domains, and the correlation between datasets. Based on the dataset attributes, we propose to measure, rank, and select datasets to build a new benchmark for model evaluation. Furthermore, a new platform for EO, termed EarthNets, is released to achieve a fair and consistent evaluation of deep learning methods on remote sensing data. EarthNets supports standard dataset libraries and cutting-edge deep learning models to bridge the gap between the remote sensing and machine learning communities. Based on this platform, extensive deep-learning methods are evaluated on the new benchmark. The insightful results are beneficial to future research. The platform and dataset collections are publicly available at https://earthnets.github.io.

Motivation & Objective

• Summarize the current status of publicly available EO datasets across tasks and domains.
• Provide a systematic, attribute-based analysis to inform dataset selection and benchmarking.
• Propose a ranking and selection method to build a large-scale, fair benchmark for RS methods.
• Release EarthNets as an open platform to enable fair, reproducible evaluations of deep learning models on EO data.

Proposed method

• Exhaustive review of 400+ public RS datasets across image classification, object detection, semantic segmentation, change detection, and other tasks.
• Extraction of ten dataset attributes per dataset (including domain, year, samples, size, classes, modality, resolution, volume, citations, links).
• Systematic analysis along five dimensions: volume, bibliometric analysis, resolution distributions, research domains, and dataset relationships.
• Ranking and filtering of five large-scale general-purpose datasets to build a new benchmark for model evaluation.
• Development of EarthNets platform to provide standard dataset libraries and support cutting-edge DL models for fair comparisons.
• Evaluation of extensive DL methods on the proposed benchmark to derive insights for future RS research.

Experimental results

Research questions

• RQ1 What are the key attributes of publicly available EO datasets across tasks and domains?
• RQ2 How can we rank and select datasets to construct a fair, large-scale benchmark for RS methods?
• RQ3 What are the relationships and correlations between different EO datasets, and how can they inform new algorithm development?
• RQ4 How can a unified platform (EarthNets) enable fair, reproducible evaluation of deep learning methods on remote sensing data?

Key findings

• Identified and categorized more than 400 public RS datasets across multiple EO tasks and domains.
• Provided ten detailed attributes for each dataset to facilitate searching and indexing.
• Proposed a ranking approach to select five large-scale general-purpose datasets and build a new RS benchmark.
• Constructed a dataset correlation matrix to offer new perspectives for cross-dataset algorithm development.
• Released EarthNets as an open platform to enable fair comparisons and efficient method development for EO tasks, with public access at earthnets.github.io.