[Paper Review] Cross-Sensor Adversarial Domain Adaptation of Landsat-8 and Proba-V images for Cloud Detection

Gonzalo Mateo‐García, Valero Laparra+2 more|arXiv (Cornell University)|Jan 1, 2020

Remote-Sensing Image ClassificationEngineering65 references34 citations

TL;DR

The paper proposes an unpaired, CycleGAN-based domain adaptation to align Proba-V with Landsat-8 upscaled data, improving cloud detection transfer between sensors.

ABSTRACT

The number of Earth observation satellites carrying optical sensors with similar characteristics is constantly growing. Despite their similarities and the potential synergies among them, derived satellite products are often developed for each sensor independently. Differences in retrieved radiances lead to significant drops in accuracy, which hampers knowledge and information sharing across sensors. This is particularly harmful for machine learning algorithms, since gathering new ground truth data to train models for each sensor is costly and requires experienced manpower. In this work, we propose a domain adaptation transformation to reduce the statistical differences between images of two satellite sensors in order to boost the performance of transfer learning models. The proposed methodology is based on the Cycle Consistent Generative Adversarial Domain Adaptation (CyCADA) framework that trains the transformation model in an unpaired manner. In particular, Landsat-8 and Proba-V satellites, which present different but compatible spatio-spectral characteristics, are used to illustrate the method. The obtained transformation significantly reduces differences between the image datasets while preserving the spatial and spectral information of adapted images, which is hence useful for any general purpose cross-sensor application. In addition, the training of the proposed adversarial domain adaptation model can be modified to improve the performance in a specific remote sensing application, such as cloud detection, by including a dedicated term in the cost function. Results show that, when the proposed transformation is applied, cloud detection models trained in Landsat-8 data increase cloud detection accuracy in Proba-V.

Motivation & Objective

• Motivate cross-sensor transfer learning for remote sensing where different satellites yield distribution shifts in radiometry and texture.
• Develop a domain adaptation pipeline that reduces inter-sensor differences without paired data.
• Enable cloud detection models trained on Landsat-8 to perform better on Proba-V imagery by learning a target-domain transformation.

Proposed method

• Upscale Landsat-8 to Proba-V spectral and spatial properties to create LU data as a training domain.
• Design an adversarial domain adaptation (A) using cycle-consistent GANs (G_PV->LU and G_LU->PV) to align Proba-V with LU domain in an unpaired setting.
• Incorporate identity, cycle, and segmentation consistency losses to enforce realistic and task-relevant translations.
• Apply a KL-divergence segmentation loss using a cloud detector trained in LU to supervise PV translations.
• Provide ground-truth datasets and pseudo-simultaneous pairs to train and evaluate the DA model without requiring paired sensors.

Experimental results

Research questions

• RQ1 Can unpaired domain adaptation transform Proba-V images to resemble Landsat-8 upscaled images to improve cloud detection transfer?
• RQ2 How much do identity, cycle, and segmentation losses contribute to the quality of the adaptation and downstream cloud detection performance?
• RQ3 Does the proposed PV->LU transformation preserve clouds and spatial-spectral information sufficiently for effective transfer?
• RQ4 What is the impact of the DA transformation on radiometric distributions and spatial frequency content of Proba-V images?
• RQ5 Can the approach work without simultaneous Landsat-8 and Proba-V pairs?

Key findings

• The PV->LU adaptation removes blue saturation and makes Proba-V radiance distributions resemble Landsat-8 upscaled data.
• Adapted PV images show higher spatial-frequency content closer to LU than raw PV, indicating improved detail transfer.
• Applying the DA transformation improves cloud detection performance of LU-trained models on PV data compared with no adaptation.
• The framework supports ablations showing the role of identity, cycle, and segmentation losses in performance.
• The method operates without paired sensor data and leverages pseudo-simultaneous datasets for training the DA model.