[Paper Review] Fine-Grained Cross-View Geo-Localization Using a Correlation-Aware Homography Estimator

Xiaolong Wang, Runsen Xu+3 more|arXiv (Cornell University)|Aug 31, 2023

Robotics and Sensor-Based Localization4 citations

TL;DR

This paper proposes HC-Net, an end-to-end method for fine-grained cross-view geo-localization that uses a geometry-constrained spherical transform to project ground-level panoramic images into a bird's-eye view, aligning them with GPS-tagged satellite images. A correlation-aware homography estimator then performs precise 2D image alignment, achieving sub-pixel accuracy and 30 FPS inference, reducing mean localization error by 21.3% and 32.4% on same-area and cross-area splits of the VIGOR benchmark, respectively.

ABSTRACT

In this paper, we introduce a novel approach to fine-grained cross-view geo-localization. Our method aligns a warped ground image with a corresponding GPS-tagged satellite image covering the same area using homography estimation. We first employ a differentiable spherical transform, adhering to geometric principles, to accurately align the perspective of the ground image with the satellite map. This transformation effectively places ground and aerial images in the same view and on the same plane, reducing the task to an image alignment problem. To address challenges such as occlusion, small overlapping range, and seasonal variations, we propose a robust correlation-aware homography estimator to align similar parts of the transformed ground image with the satellite image. Our method achieves sub-pixel resolution and meter-level GPS accuracy by mapping the center point of the transformed ground image to the satellite image using a homography matrix and determining the orientation of the ground camera using a point above the central axis. Operating at a speed of 30 FPS, our method outperforms state-of-the-art techniques, reducing the mean metric localization error by 21.3% and 32.4% in same-area and cross-area generalization tasks on the VIGOR benchmark, respectively, and by 34.4% on the KITTI benchmark in same-area evaluation.

Motivation & Objective

• To address the limitations of coarse cross-view localization methods that suffer from tens-of-meters errors due to satellite patch segmentation.
• To overcome the high computational cost and low resolution of sampling-based and descriptor-splitting methods in fine-grained geo-localization.
• To enable accurate, real-time localization without requiring prior pose estimates or extensive data augmentation.
• To reduce dependency on dense matching point supervision by introducing a correlation-aware homography estimation module.
• To improve generalization across diverse urban environments, including unseen cities, without retraining.

Proposed method

• A differentiable spherical transform is applied to ground-level panoramic images to project them into a bird’s-eye view (BEV), aligning the perspective with satellite images using the ground camera’s imaging model.
• The spherical transform enables direct 2D image alignment between the transformed ground image and the satellite image, reducing the problem to a geometric alignment task.
• A correlation-aware homography estimator uses a recurrent convolutional neural network to maximize correlation between similar regions in feature maps while suppressing unobservable or occluded areas.
• The method avoids iterative optimization by directly learning the homography matrix end-to-end, minimizing reliance on four-point matching supervision.
• The network outputs the homography matrix, which maps the center of the BEV to the satellite image, enabling precise GPS location and yaw estimation.
• A Pseudo-Siamese backbone with non-shared weights processes ground and satellite images separately, improving feature representation learning.

Experimental results

Research questions

• RQ1 Can a differentiable spherical transform effectively bridge the domain gap between ground and aerial views for cross-view localization?
• RQ2 Can a correlation-aware homography estimator outperform traditional feature-based methods in occluded or low-overlap scenarios?
• RQ3 Does the proposed method achieve higher accuracy and faster inference than state-of-the-art approaches in both same-area and cross-area generalization?
• RQ4 Can the model generalize to unseen cities without retraining, as demonstrated on the CVUSA dataset?
• RQ5 To what extent does the absence of four-point matching supervision affect homography estimation accuracy?

Key findings

• HC-Net reduces the mean localization error by 21.3% on the same-area split and 32.4% on the cross-area split of the VIGOR benchmark compared to state-of-the-art methods.
• On the KITTI benchmark, the method achieves a 34.4% lower mean error than CCVPE in same-area evaluation with a ±10° orientation prior.
• The model runs at 30 FPS, demonstrating high computational efficiency with only 11.21M parameters and 1900 MiB memory usage.
• Abllation studies confirm that the Pseudo-Siamese backbone reduces mean error by 0.71m compared to shared weights.
• Replacing the homography module with SuperGlue or LoFTR increases mean error to 13.06m and 28.85m, respectively, proving the superiority of the proposed module.
• The model generalizes well to unseen cities, successfully aligning BEV projections from CVUSA with satellite images without retraining.