[Paper Review] Artificial Generation of Big Data for Improving Image Classification: A Generative Adversarial Network Approach on SAR Data

Dimitrios Marmanis, Wei Yao+6 more|arXiv (Cornell University)|Nov 6, 2017

Synthetic Aperture Radar (SAR) Applications and Techniques8 references17 citations

TL;DR

This paper introduces the first large-scale, fully-annotated SAR image dataset with over 60,000 patches across seven urban/peri-urban classes, and evaluates deep learning for SAR classification using ResNet-50, achieving 93.2% accuracy. It further explores GAN-based synthetic data generation via a modified BEGAN model to augment training, though initial results show no improvement in classification performance despite realistic synthetic outputs.

ABSTRACT

Very High Spatial Resolution (VHSR) large-scale SAR image databases are still an unresolved issue in the Remote Sensing field. In this work, we propose such a dataset and use it to explore patch-based classification in urban and periurban areas, considering 7 distinct semantic classes. In this context, we investigate the accuracy of large CNN classification models and pre-trained networks for SAR imaging systems. Furthermore, we propose a Generative Adversarial Network (GAN) for SAR image generation and test, whether the synthetic data can actually improve classification accuracy.

Motivation & Objective

• To address the scarcity of large-scale, fully-annotated SAR image datasets for deep learning in remote sensing.
• To evaluate the performance of deep CNNs, particularly ResNet-50, on very high-resolution SAR data for patch-based classification.
• To investigate whether synthetic SAR images generated via GANs can improve classification accuracy.
• To explore transfer learning from optical pre-trained models and assess their relevance to SAR image classification.
• To develop and adapt a GAN-based approach (BEGAN) for realistic SAR image synthesis at scale.

Proposed method

• Construction of a large-scale SAR image dataset from 288 TerraSAR-X scenes across six continents, using HH-polarized, 2.9 m resolution, 160×160 pixel patches.
• Training of a ResNet-50 classifier from scratch on the real dataset, with final layers adapted to 7-class classification.
• Adaptation of the BEGAN GAN architecture for SAR image generation, tested in three scenarios: full-size (160×160), intermediate (80×80 with downsampling), and simple (80×80 with cropping).
• Synthetic patches from the 'simple' scenario were upsampled to 160×160 and added to the real training set for fine-tuning the classifier.
• Evaluation of classification accuracy before and after data augmentation using synthetic SAR images.
• Use of 8-bit intensity conversion and spatial cropping/downsampling to standardize input for GAN and classifier training.

Experimental results

Research questions

• RQ1 Can a large-scale, fully-annotated SAR image dataset enable high-performance deep learning classification?
• RQ2 Does fine-tuning a pre-trained ResNet-50 on SAR data yield superior performance compared to transfer learning from optical pre-trained models?
• RQ3 Can GAN-generated synthetic SAR images effectively augment real training data to improve classification accuracy?
• RQ4 What are the limitations of current GAN architectures in generating realistic SAR image patches at scale?
• RQ5 How does the spatial context and resolution mismatch in synthetic data affect downstream classification performance?

Key findings

• The proposed dataset of over 60,000 SAR image patches enables training of deep CNNs with state-of-the-art performance, achieving 93.2% classification accuracy on seven semantic classes.
• Pre-training with optical image weights provided little benefit, indicating significant domain shift between optical and SAR image statistics.
• The BEGAN-based GAN successfully generated realistic-looking SAR patches in the 'simple' scenario (80×80 cropped patches), though with reduced spatial context.
• Despite generating visually plausible synthetic images, adding 5,100 synthetic patches (mainly for the 'Settlement' class) did not improve or degrade the classifier’s 93.2% accuracy.
• The failure to improve performance suggests that current synthetic data may lack sufficient diversity or semantic fidelity to benefit the classifier, despite visual realism.
• The study confirms that deep learning models can achieve high accuracy on SAR data when sufficient labeled data is available, setting a new benchmark for patch-based SAR classification.