[Paper Review] Tracking Urbanization in Developing Regions with Remote Sensing Spatial-Temporal Super-Resolution

Yutong He, William Zhang+4 more|arXiv (Cornell University)|Apr 4, 2022

Advanced Vision and Imaging8 citations

TL;DR

This paper proposes a spatial-temporal super-resolution pipeline that combines a single high-resolution (HR) satellite image with a time series of low-resolution (LR) images to generate accurate, high-resolution time series for tracking urban construction in developing regions. By leveraging temporal consistency and HR detail, the method significantly improves object tracker performance over baselines using only LR images or single-image super-resolution, achieving near-HR performance on object tracking tasks.

ABSTRACT

Automated tracking of urban development in areas where construction information is not available became possible with recent advancements in machine learning and remote sensing. Unfortunately, these solutions perform best on high-resolution imagery, which is expensive to acquire and infrequently available, making it difficult to scale over long time spans and across large geographies. In this work, we propose a pipeline that leverages a single high-resolution image and a time series of publicly available low-resolution images to generate accurate high-resolution time series for object tracking in urban construction. Our method achieves significant improvement in comparison to baselines using single image super-resolution, and can assist in extending the accessibility and scalability of building construction tracking across the developing world.

Motivation & Objective

• To address the challenge of tracking urbanization in developing regions where high-resolution (HR) satellite imagery is scarce and expensive to acquire.
• To improve the scalability and accessibility of building construction tracking by combining the temporal abundance of low-resolution (LR) imagery with the spatial detail of a single HR image.
• To develop a pipeline that enhances object tracker performance in regions lacking HR data and labeled construction footprints.

Proposed method

• The method uses a spatial-temporal super-resolution model (G) that takes a time series of LR images and a single HR reference image to generate a synthetic HR time series.
• The model is trained with a combination of adversarial loss, perceptual loss (LPIPS), and a novel edge-aware loss (EAD) to preserve structural and boundary details.
• The generated HR time series are fed into a pre-trained object tracker (e.g., YOLO or similar) to detect and track building constructions over time.
• The framework operates under a weakly supervised setting: only the most recent HR image is available at inference time, while the model learns from paired HR and LR sequences during training.
• The method leverages temporal coherence across the LR time series to stabilize and refine the super-resolution output, improving fidelity and consistency.
• The approach is evaluated on a custom dataset aligned with SpaceNet 7, using Landsat 8 (30m GSD) as the LR source and high-resolution imagery as the HR reference.

Experimental results

Research questions

• RQ1 Can a spatial-temporal super-resolution model effectively generate high-resolution satellite images from a single HR reference and a time series of LR images to improve object tracking?
• RQ2 How does the performance of object tracking using the generated HR images compare to tracking on real HR images, LR images, and images generated by single-image super-resolution?
• RQ3 To what extent does incorporating a perceptual and edge-aware loss improve the quality of super-resolved images for downstream object tracking tasks?
• RQ4 Can this pipeline be deployed in resource-constrained regions lacking access to continuous HR satellite data while maintaining high tracking accuracy?

Key findings

• The proposed method achieves object tracking performance on generated images that is comparable to ground truth HR images, with a mean Intersection over Union (IoU) of 0.586 in the EAD+LPIPS setting.
• Spatial-temporal super-resolution significantly outperforms single-image super-resolution and direct LR input, with IoU increasing from 0.471 (LR) to 0.586 (Ours with EAD+LPIPS) on the target domain.
• The inclusion of the LPIPS loss in the EAD+LPIPS variant leads to a 0.015 improvement in IoU over EAD alone, demonstrating the benefit of perceptual fidelity.
• The method enables detection of small building constructions that are invisible in LR images but clearly visible in the generated HR outputs.
• In both training settings—using ImageNet and SpaceNet 7 pretraining—the method consistently outperforms baselines across all metrics, including accuracy and FWIoU.
• The model maintains faithful tracking results that align closely with ground truth, even when the generated images are not perfect, indicating robustness to reconstruction errors.