[Paper Review] OpenContrails: Benchmarking Contrail Detection on GOES-16 ABI

Joe Yue-Hei Ng, Kevin McCloskey+7 more|arXiv (Cornell University)|Apr 4, 2023

Air Traffic Management and Optimization20 citations

TL;DR

This paper creates OpenContrails, a human-labeled GOES-16 ABI contrail dataset, and presents a multi-frame CNN model that uses temporal context to improve contrail detection and line-segment outputs for benchmarking.

ABSTRACT

Contrails (condensation trails) are line-shaped ice clouds caused by aircraft and are likely the largest contributor of aviation-induced climate change. Contrail avoidance is potentially an inexpensive way to significantly reduce the climate impact of aviation. An automated contrail detection system is an essential tool to develop and evaluate contrail avoidance systems. In this paper, we present a human-labeled dataset named OpenContrails to train and evaluate contrail detection models based on GOES-16 Advanced Baseline Imager (ABI) data. We propose and evaluate a contrail detection model that incorporates temporal context for improved detection accuracy. The human labeled dataset and the contrail detection outputs are publicly available on Google Cloud Storage at gs://goes_contrails_dataset.

Motivation & Objective

• Provide a public, high-quality, per-pixel contrail labeling dataset on GOES-16 ABI to enable reproducible research and model benchmarking.
• Develop a neural network model that leverages temporal context to improve contrail detection accuracy.
• Enable conversion of pixel-wise detections into line segments for contrail characterization and flight attribution.
• Offer evaluation metrics and baseline results to facilitate comparison and benchmarking across methods.

Proposed method

• Assemble a public dataset (OpenContrails) of human-labeled contrail masks on GOES-16 ABI imagery with 256×256 patches and 5-frame context for labeling.
• Train a single-frame semantic segmentation model based on DeeplabV3+ with a dilated ResNet backbone and SE blocks.
• Extend to a multi-frame spatio-temporal model using an inflated 3D ResNet (I3D) style backbone with temporal convolutions to incorporate multiple frames.
• Use an ASPP decoder and post-process pixel masks with a LineSegmentDetector to produce contrail line segments.
• Pre-train on ImageNet for the single-frame model and on Kinetics-400 for the multi-frame backbone; apply data augmentation and cosine learning rate scheduling.
• Evaluate using per-pixel AUC-PR and per-contrail precision/recall curves; compare against Meijer et al.’s model; convert binary masks to lines for evaluation.

Experimental results

Research questions

• RQ1 Can a publicly labeled contrail dataset on GOES-16 ABI be created with high inter-labeler agreement?
• RQ2 Does incorporating temporal context through multi-frame inputs improve contrail detection performance over single-frame models?
• RQ3 How well can pixel-wise contrail detections be translated into accurate line segments for contrail characterization?
• RQ4 What are the baseline performance and generalization capabilities of GOES-16 contrail detectors across resolutions, temporal contexts, and spatial regions?

Key findings

• The OpenContrails dataset provides 20,544 training and 1,866 validation examples with contrail pixels around 1.2% of training pixels.
• The multi-frame model consistently outperforms single-frame models in pixel-wise AUC-PR and in per-contrail precision/recall.
• Larger input resolutions and more temporal context generally improve pixel-level detection performance, with diminishing returns at very high resolutions or frame counts.
• The detector achieves higher precision/recall than a prior Landsat-8 based approach when evaluated on comparable tasks (e.g., 60% recall with substantially higher precision).
• Contrail coverage in CONUS during 2018–2019 is around 0.19% on average, with diurnal and seasonal patterns consistent with prior work, and flight-density clustering along major routes.