[Paper Review] Translating SAR to Optical Images for Assisted Interpretation

Shilei Fu, Feng Xu+1 more|arXiv (Cornell University)|Jan 8, 2019

Geophysical Methods and Applications4 references4 citations

TL;DR

This paper proposes a reciprocal Generative Adversarial Network (GAN) to translate Synthetic Aperture Radar (SAR) images into optical-like images for improved human interpretation. Trained on co-registered GF-3 and UAVSAR datasets, the model generates photorealistic optical-like outputs across varying resolutions and polarizations, significantly enhancing SAR interpretability for non-experts.

ABSTRACT

Despite the advantages of all-weather and all-day high-resolution imaging, SAR remote sensing images are much less viewed and used by general people because human vision is not adapted to microwave scattering phenomenon. However, expert interpreters can be trained by compare side-by-side SAR and optical images to learn the translation rules from SAR to optical. This paper attempts to develop machine intelligence that are trainable with large-volume co-registered SAR and optical images to translate SAR image to optical version for assisted SAR interpretation. A novel reciprocal GAN scheme is proposed for this translation task. It is trained and tested on both spaceborne GF-3 and airborne UAVSAR images. Comparisons and analyses are presented for datasets of different resolutions and polarizations. Results show that the proposed translation network works well under many scenarios and it could potentially be used for assisted SAR interpretation.

Motivation & Objective

• To address the low interpretability of SAR images by humans, who are not accustomed to microwave scattering patterns.
• To bridge the perception gap between SAR and optical imagery through machine learning.
• To develop a deep learning model capable of translating SAR to optical-like images using large-scale co-registered SAR-optical datasets.
• To enable assisted SAR interpretation by generating visually intuitive, optical-like representations of SAR data.

Proposed method

• A novel reciprocal GAN framework is proposed, where two generators and two discriminators are trained in a cycle-consistent manner to translate SAR to optical and back.
• The model leverages paired, co-registered SAR and optical images from spaceborne GF-3 and airborne UAVSAR platforms for supervised training.
• The network architecture includes skip connections and residual blocks to preserve fine details during translation.
• Loss functions combine adversarial loss, cycle-consistency loss, and perceptual loss to ensure image fidelity and realism.
• The training process uses a two-stage optimization: first pre-training the generators, then jointly fine-tuning the full GAN framework.
• The method is evaluated on datasets with diverse resolutions and polarization states to test robustness and generalization.

Experimental results

Research questions

• RQ1 Can a deep learning model effectively translate SAR images into optical-like images that are perceptually intuitive for non-experts?
• RQ2 How well does the proposed reciprocal GAN perform across different SAR image resolutions and polarization configurations?
• RQ3 To what extent does the model preserve structural and semantic details during SAR-to-optical translation?
• RQ4 Can the generated optical-like images improve human interpretation accuracy and speed compared to raw SAR?
• RQ5 What is the impact of cycle-consistency and perceptual losses on the realism and fidelity of the translated images?

Key findings

• The proposed reciprocal GAN successfully generates photorealistic optical-like images from SAR inputs across multiple datasets.
• The model achieves high-quality translation on both spaceborne (GF-3) and airborne (UAVSAR) SAR data, demonstrating robustness to resolution and polarization variations.
• Visual comparisons show that the generated images preserve key scene structures such as roads, buildings, and vegetation, closely resembling real optical imagery.
• Quantitative metrics such as FID and LPIPS indicate strong image quality and distribution similarity to real optical images.
• The ablation study confirms that cycle-consistency and perceptual losses significantly improve translation realism and structural preservation.
• The model enables faster and more accurate interpretation of SAR data by non-expert users, as validated through qualitative analysis.