[Paper Review] CG-Net: Conditional GIS-aware Network for Individual Building Segmentation in VHR SAR Images

Yao Sun, Yuansheng Hua+2 more|arXiv (Cornell University)|Nov 17, 2020

Automated Road and Building Extraction72 references45 citations

TL;DR

This paper proposes CG-Net, a conditional GIS-aware deep learning framework that leverages building footprints from GIS data to improve individual building segmentation in very high resolution (VHR) SAR images. By conditioning feature normalization on GIS-derived footprints, CG-Net achieves state-of-the-art performance with a 75.08% F1 score on a TerraSAR-X image of Berlin, demonstrating robustness to GIS positioning errors and enabling accurate 3D LoD1 building model reconstruction from SAR data.

ABSTRACT

Object retrieval and reconstruction from very high resolution (VHR) synthetic aperture radar (SAR) images are of great importance for urban SAR applications, yet highly challenging owing to the complexity of SAR data. This paper addresses the issue of individual building segmentation from a single VHR SAR image in large-scale urban areas. To achieve this, we introduce building footprints from GIS data as complementary information and propose a novel conditional GIS-aware network (CG-Net). The proposed model learns multi-level visual features and employs building footprints to normalize the features for predicting building masks in the SAR image. We validate our method using a high resolution spotlight TerraSAR-X image collected over Berlin. Experimental results show that the proposed CG-Net effectively brings improvements with variant backbones. We further compare two representations of building footprints, namely complete building footprints and sensor-visible footprint segments, for our task, and conclude that the use of the former leads to better segmentation results. Moreover, we investigate the impact of inaccurate GIS data on our CG-Net, and this study shows that CG-Net is robust against positioning errors in GIS data. In addition, we propose an approach of ground truth generation of buildings from an accurate digital elevation model (DEM), which can be used to generate large-scale SAR image datasets. The segmentation results can be applied to reconstruct 3D building models at level-of-detail (LoD) 1, which is demonstrated in our experiments.

Motivation & Objective

• To address the challenge of individual building segmentation in large-scale VHR SAR images, where layover and shadowing cause severe overlapping and ambiguity.
• To improve segmentation accuracy by integrating complementary GIS building footprint data as conditional supervision.
• To evaluate the impact of different GIS footprint representations—complete footprints versus sensor-visible segments—on segmentation performance.
• To assess robustness of the method under realistic GIS data inaccuracies, such as positioning errors.
• To demonstrate the utility of segmentation results for reconstructing 3D LoD1 building models from SAR data.

Proposed method

• CG-Net employs a conditional feature normalization module that uses GIS-derived building footprints to guide feature learning in a U-Net-style encoder-decoder network.
• The model fuses multi-level visual features from a deep CNN backbone (e.g., DeepLabv3+) with spatially aligned GIS footprint data to refine segmentation masks.
• A novel ground truth generation pipeline uses high-resolution DEM data to automatically generate accurate building masks, enabling large-scale SAR dataset creation.
• The framework is trained using a combination of SAR patches and corresponding GIS footprints, with loss functions optimized for IoU and F1 score.
• The method supports both complete building footprints and partial, sensor-visible footprint segments as input, enabling flexibility in data availability.
• 3D LoD1 models are reconstructed by estimating building height from layover length in the slant range direction using the formula h = l / cosθ.

Experimental results

Research questions

• RQ1 Can GIS-derived building footprints significantly improve individual building segmentation in VHR SAR images compared to purely data-driven methods?
• RQ2 Which representation of GIS footprints—complete footprints or sensor-visible segments—yields better segmentation performance?
• RQ3 How robust is the proposed CG-Net to positioning errors commonly found in open-source GIS data like OpenStreetMap?
• RQ4 Can the segmentation results be effectively leveraged for 3D building model reconstruction at LoD1?
• RQ5 What is the achievable accuracy of building height estimation from SAR-derived layover regions?

Key findings

• CG-Net achieves a state-of-the-art F1 score of 75.08% on the Berlin TerraSAR-X dataset, outperforming baseline models.
• Using complete building footprints as input leads to significantly better segmentation results than using only sensor-visible footprint segments, with a 4.5% improvement in IoU.
• The model remains robust to GIS positioning errors, maintaining strong performance even when footprints are degraded by up to 10m error, as shown by a 3.62% drop in F1 score when using erroneous data.
• The average building height error in LoD1 reconstruction is 2.39 meters when compared to ground truth from a high-resolution DEM.
• The proposed ground truth generation method using DEM enables scalable, accurate annotation of large-scale SAR images for training deep learning models.
• Visual results confirm that buildings with large layover areas (visible in SAR) are correctly identified as high-rise, while those with small layover are reconstructed as low-rise, matching real-world geometry.