[Paper Review] DeepMask: an algorithm for cloud and cloud shadow detection in optical satellite remote sensing images using deep residual network

Ke Xu, Kaiyu Guan+3 more|arXiv (Cornell University)|Nov 9, 2019

Remote Sensing in Agriculture46 references17 citations

TL;DR

DeepMask is a deep learning-based cloud and cloud shadow detection algorithm using a residual neural network (ResNet) for pixel-level segmentation in optical satellite imagery. It achieves 93.56% accuracy across land types—significantly outperforming CFMask (85.36%)—and maintains high performance with only four spectral bands, enabling broad applicability to diverse satellite platforms without thermal bands.

ABSTRACT

Detecting and masking cloud and cloud shadow from satellite remote sensing images is a pervasive problem in the remote sensing community. Accurate and efficient detection of cloud and cloud shadow is an essential step to harness the value of remotely sensed data for almost all downstream analysis. DeepMask, a new algorithm for cloud and cloud shadow detection in optical satellite remote sensing imagery, is proposed in this study. DeepMask utilizes ResNet, a deep convolutional neural network, for pixel-level cloud mask generation. The algorithm is trained and evaluated on the Landsat 8 Cloud Cover Assessment Validation Dataset distributed across 8 different land types. Compared with CFMask, the most widely used cloud detection algorithm, land-type-specific DeepMask models achieve higher accuracy across all land types. The average accuracy is 93.56%, compared with 85.36% from CFMask. DeepMask also achieves 91.02% accuracy on all-land-type dataset. Compared with other CNN-based cloud mask algorithms, DeepMask benefits from the parsimonious architecture and the residual connection of ResNet. It is compatible with input of any size and shape. DeepMask still maintains high performance when using only red, green, blue, and NIR bands, indicating its potential to be applied to other satellite platforms that only have limited optical bands.

Motivation & Objective

• To develop a highly accurate, automated cloud and cloud shadow detection algorithm for optical satellite remote sensing images.
• To overcome the limitations of threshold-based and traditional machine learning methods, which struggle with bright surfaces and optically thin clouds.
• To leverage deep convolutional neural networks to capture spatial, textural, and spectral features for improved detection.
• To create a flexible, efficient, and scalable algorithm compatible with various satellite platforms, including those with limited spectral bands.
• To reduce reliance on pre-processing or post-processing steps while maintaining high precision and generalization across diverse land cover types.

Proposed method

• The algorithm uses a deep residual network (ResNet) as the backbone for end-to-end pixel-level classification of cloud and cloud shadow.
• It processes a 15×15 pixel local region around each central pixel to capture contextual spatial and textural information.
• The model is trained using the Landsat 8 Cloud Cover Assessment (CCA) validation dataset, covering eight distinct land cover types.
• Input images can be of arbitrary size and shape, enabling flexible deployment across different satellite data formats.
• The algorithm does not require thermal bands or fixed-size inputs, enhancing compatibility with diverse optical sensors.
• Performance is tuned via a confidence threshold to balance precision and recall, allowing user-defined trade-offs.

Experimental results

Research questions

• RQ1 Can a deep learning-based approach outperform traditional threshold-based cloud detection methods like CFMask in accuracy and robustness?
• RQ2 To what extent can a ResNet-based model generalize across diverse land cover types without requiring land-type-specific tuning?
• RQ3 How well does the algorithm perform when using only four spectral bands (RGB and NIR), and can it be applied to satellites with limited spectral capabilities?
• RQ4 Can the model maintain high performance without pre-processing or post-processing steps, such as superpixel segmentation or morphological filtering?
• RQ5 How does the model handle challenging cases such as snow/ice, bright surfaces, and optically thin clouds, which are known to confuse existing algorithms?

Key findings

• The land-type-specific DeepMask model achieves an average accuracy of 93.56% across all eight land cover types, significantly outperforming CFMask’s 85.36%.
• The all-land-type DeepMask model achieves 91.02% accuracy, still substantially higher than CFMask’s performance.
• DeepMask maintains high accuracy even when trained on only four spectral bands (red, green, blue, and NIR), indicating strong potential for use with platforms lacking thermal bands.
• The algorithm shows minimal performance degradation when spectral bands are dropped, demonstrating robustness to input variability.
• DeepMask successfully distinguishes clouds and shadows from snow/ice and other bright surfaces, where CFMask often fails.
• The model’s local region-based inference mechanism enables high accuracy without requiring fixed-size inputs or complex pre/post-processing pipelines.