[Paper Review] Building Detection from Satellite Images on a Global Scale

Amy Zhang, Xianming Liu+2 more|arXiv (Cornell University)|Jul 27, 2017

Remote-Sensing Image Classification5 references19 citations

TL;DR

This paper presents a globally scalable building detection system using weakly supervised and semi-supervised deep learning on high-resolution satellite imagery, achieving 90% precision and 89% recall. It introduces a dual-model pipeline combining a fast classification network with a weakly supervised segmentation head, enhanced by denoising and active learning, enabling large-scale population density mapping across diverse global regions with minimal manual annotation.

ABSTRACT

In the last several years, remote sensing technology has opened up the possibility of performing large scale building detection from satellite imagery. Our work is some of the first to create population density maps from building detection on a large scale. The scale of our work on population density estimation via high resolution satellite images raises many issues, that we will address in this paper. The first was data acquisition. Labeling buildings from satellite images is a hard problem, one where we found our labelers to only be about 85% accurate at. There is a tradeoff of quantity vs. quality of labels, so we designed two separate policies for labels meant for training sets and those meant for test sets, since our requirements of the two set types are quite different. We also trained weakly supervised footprint detection models with the classification labels, and semi-supervised approaches with a small number of pixel-level labels, which are very expensive to procure.

Motivation & Objective

• To develop a scalable, global building detection system capable of processing high-resolution satellite imagery across diverse geographic and architectural contexts.
• To address the challenge of imbalanced data and high annotation costs by employing weakly supervised and semi-supervised learning with active learning.
• To reduce systematic errors from image quality variations (e.g., cloud cover, noise) through end-to-end denoising and data quality filtering.
• To improve model generalization across countries by training a global model with incremental fine-tuning on select regions.
• To enable large-scale population density estimation by generating accurate, open-source building footprint maps at global scale.

Proposed method

• A weakly supervised SegNet-based classification model uses global average pooling on 64×64 image masks to produce pixel-level probability maps, trained with weak 'pixel-level' labels.
• A bounding box proposal algorithm based on edge detection reduces the number of candidate regions by 50%, focusing computation on likely building areas.
• A semi-supervised approach leverages a small number of expensive pixel-level annotations to improve segmentation performance.
• An end-to-end shallow neural network is trained to denoise satellite images, addressing noise and cloud occlusion issues before classification.
• Active learning selects uncertain masks (near classification threshold) for labeling, improving data efficiency and model robustness.
• A feedback loop visualizes results via KMZ files overlaid on Google Earth to detect and correct systematic errors.

Experimental results

Research questions

• RQ1 Can a single global deep learning model achieve high-precision building detection across diverse global regions with varying building styles and image quality?
• RQ2 How can weakly supervised and semi-supervised learning reduce the need for costly pixel-level annotations in large-scale building detection?
• RQ3 To what extent can data denoising and quality filtering improve model robustness in the presence of cloud cover and imaging artifacts?
• RQ4 How does incremental fine-tuning on specific countries improve a global model’s performance without overfitting?
• RQ5 What role does active learning play in balancing dataset class distribution and improving model confidence?

Key findings

• The global building detection system achieved 90% precision and 89% recall on a highly imbalanced test set where 93% of samples were non-building.
• The use of a bounding box proposal algorithm reduced the number of candidate masks by 50%, significantly accelerating inference while maintaining high recall.
• Denoising with a learned neural network improved classification results on low-quality images, as shown in visual comparisons before and after denoising.
• Fine-tuning the global model on a few target countries led to 20–40% gains in precision and recall on validation sets, demonstrating effective transfer learning.
• Active learning and uncertainty sampling improved model performance by prioritizing ambiguous samples for labeling, enhancing data efficiency.
• The system successfully generated high-resolution building heat maps for Mozambique, Madagascar, and India, enabling improved population density estimation compared to prior methods.