[Paper Review] Mapping Temporary Slums from Satellite Imagery using a Semi-Supervised Approach

Mohibur Rehman, Izza Aftab+2 more|arXiv (Cornell University)|Apr 9, 2022

Remote-Sensing Image Classification19 references20 citations

TL;DR

This paper proposes a semi-supervised deep learning framework that automatically detects temporary slums in satellite imagery using minimal labeled seed data. By leveraging temporal changes in urban structures to identify initial seed images, the method iteratively selects high-confidence unlabeled images via embedding similarity and pseudo-labeling, achieving a 1.9x improvement over semi-supervised baselines and 1.8x over supervised baselines on a new dataset of 2.28 km² across 12 Pakistani cities.

ABSTRACT

One billion people worldwide are estimated to be living in slums, and documenting and analyzing these regions is a challenging task. As compared to regular slums; the small, scattered and temporary nature of temporary slums makes data collection and labeling tedious and time-consuming. To tackle this challenging problem of temporary slums detection, we present a semi-supervised deep learning segmentation-based approach; with the strategy to detect initial seed images in the zero-labeled data settings. A small set of seed samples (32 in our case) are automatically discovered by analyzing the temporal changes, which are manually labeled to train a segmentation and representation learning module. The segmentation module gathers high dimensional image representations, and the representation learning module transforms image representations into embedding vectors. After that, a scoring module uses the embedding vectors to sample images from a large pool of unlabeled images and generates pseudo-labels for the sampled images. These sampled images with their pseudo-labels are added to the training set to update the segmentation and representation learning modules iteratively. To analyze the effectiveness of our technique, we construct a large geographically marked dataset of temporary slums. This dataset constitutes more than 200 potential temporary slum locations (2.28 square kilometers) found by sieving sixty-eight thousand images from 12 metropolitan cities of Pakistan covering 8000 square kilometers. Furthermore, our proposed method outperforms several competitive semi-supervised semantic segmentation baselines on a similar setting. The code and the dataset will be made publicly available.

Motivation & Objective

• Address the challenge of detecting small, scattered, and temporary slums due to lack of labeled data and geographic information.
• Develop a method to initiate detection from zero labeled data using temporal changes in satellite imagery.
• Construct a large, geographically marked dataset of temporary slums across 12 metropolitan cities in Pakistan.
• Enable scalable, automated detection of temporary slums in low-resource settings using minimal human annotation.

Proposed method

• Use temporal consistency in satellite imagery to automatically identify a small set of initial seed images containing temporary slums.
• Train a U-Net-based segmentation model and a 64-dimensional embedding model from the initial seed data.
• Compute cosine similarity between labeled and unlabeled image embeddings to score and select the most similar images for pseudo-labeling.
• Apply pixel-level filtering: remove images with predicted slum areas below 5% or above 5% for non-slum images to ensure quality.
• Iteratively add high-confidence pseudo-labeled images to the training set and retrain the segmentation and embedding models.
• Stop iteration when the slum-to-non-slum ratio in the expanded training set matches the initial ratio, ensuring balanced learning.

Experimental results

Research questions

• RQ1 Can a semi-supervised approach detect temporary slums with minimal initial labeled data?
• RQ2 How can temporal changes in satellite imagery be exploited to discover initial seed images for slum detection?
• RQ3 What is the performance gain of the proposed method over existing semi-supervised and supervised baselines in low-data regimes?
• RQ4 Can the framework be generalized to other urban regions with minimal adaptation?
• RQ5 How effective is the iterative pseudo-labeling strategy using embedding similarity for improving segmentation accuracy?

Key findings

• The proposed method achieves a mean Intersection over Union (mIoU) of 0.33, outperforming the supervised baseline (mIoU 0.17) by 1.8 times.
• The method improves upon competitive semi-supervised baselines such as Cutmix, Cutout, ICT, and VAT, achieving a 1.9x improvement in mIoU.
• Ablation studies show that K=30 yields the best performance (mIoU=0.33), with K=50 and K=100 showing slightly lower mIoU values.
• The method successfully identifies 345 slum images from an unlabeled pool of 3938 images, reaching the target slum-to-non-slum ratio after iterative refinement.
• The framework enables detection from a zero-dataset scenario, with only 32 manually labeled seed images required to initiate the process.
• The constructed dataset spans 2.28 km² across 12 metropolitan cities in Pakistan, providing a valuable benchmark for future research.