[Paper Review] Spatial Morphing Kernel Regression For Feature Interpolation

Xueqing Deng, Yi Zhu+1 more|arXiv (Cornell University)|Feb 21, 2018

Advanced Image and Video Retrieval Techniques12 references3 citations

TL;DR

This paper proposes spatial morphing kernel regression to interpolate high-dimensional features from sparse, geotagged social media images for dense geographic classification. By incorporating prior knowledge of spatial boundaries via graph Laplacians, the method improves interpolation accuracy and reduces spurious classifications, outperforming standard methods like IDW and Gaussian kernels in both synthetic and real-world land use mapping tasks in San Francisco.

ABSTRACT

In recent years, geotagged social media has become popular as a novel source for geographic knowledge discovery. Ground-level images and videos provide a different perspective than overhead imagery and can be applied to a range of applications such as land use mapping, activity detection, pollution mapping, etc. The sparse and uneven distribution of this data presents a problem, however, for generating dense maps. We therefore investigate the problem of spatially interpolating the high-dimensional features extracted from sparse social media to enable dense labeling using standard classifiers. Further, we show how prior knowledge about region boundaries can be used to improve the interpolation through spatial morphing kernel regression. We show that an interpolate-then-classify framework can produce dense maps from sparse observations but that care must be taken in choosing the interpolation method. We also show that the spatial morphing kernel improves the results.

Motivation & Objective

• To address the challenge of creating dense geographic maps from sparse, unevenly distributed geotagged social media images.
• To investigate the effectiveness of different interpolation methods for high-dimensional features extracted from ground-level images.
• To integrate prior knowledge of spatial heterogeneity—such as region boundaries—into the interpolation process to improve accuracy.
• To evaluate the proposed method on both synthetic data with known ground truth and real-world data from San Francisco.
• To demonstrate that spatial morphing kernel regression yields more accurate and stable classification maps than conventional interpolation techniques.

Proposed method

• A pre-trained ResNet-101 CNN is used to extract 2,048-dimensional features from ground-level images at sparse locations.
• The framework follows an 'interpolate-then-classify' pipeline: features are interpolated over a dense grid before classification using the CNN's fully connected layer.
• Inverse distance weighting (IDW) and Gaussian kernel regression are used as baseline interpolation methods, assuming spatial smoothness.
• Spatial morphing kernels (SMSK and SMMK) are introduced, using graph Laplacians constructed from either a mesh grid or actual sparse feature locations to model spatial heterogeneity.
• The graph Laplacian encodes prior knowledge of discontinuities and boundaries, such as building or land parcel edges, to guide interpolation.
• Two variants of spatial morphing kernels are evaluated: SMSK (using sparse feature locations) and SMMK (using a mesh grid), both modulated by a parameter γ=100.

Experimental results

Research questions

• RQ1 How does the choice of interpolation method affect the quality of dense geographic classification from sparse social media features?
• RQ2 Can incorporating prior spatial knowledge—such as administrative or physical boundaries—improve the accuracy of feature interpolation?
• RQ3 How do spatial morphing kernels compare to standard kernel methods like IDW and Gaussian regression in terms of boundary accuracy and noise reduction?
• RQ4 Does the performance of spatial morphing kernels depend on data sparsity, and if so, how?
• RQ5 Can the proposed interpolate-then-classify framework produce reliable dense maps on real-world data where ground truth is unknown?

Key findings

• Spatial morphing kernel regression (SMSK and SMMK) significantly reduces spurious class assignments compared to IDW and Gaussian kernels, with SMSK introducing no spurious classes in the two-image-per-region case.
• On synthetic data, SMSK and SMMK outperform non-spatially-aware methods, achieving higher mean Intersection over Union (mIoU) across all image densities.
• For sparse configurations (e.g., one image per region), SMSK outperforms SMMK in mIoU and noise reduction, indicating better stability under low data density.
• On real-world San Francisco data, SMMK produces the most accurate and boundary-consistent maps, with fewer artifacts near parcel boundaries.
• The use of spatial morphing kernels leads to more realistic and geographically plausible classification maps, especially near region discontinuities.
• The framework demonstrates that integrating spatial prior knowledge through graph Laplacians enhances interpolation fidelity and classifier robustness.