[论文解读] Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching

Yujiao Shi, Xin Yu+2 more|arXiv (Cornell University)|May 8, 2020

Advanced Image and Video Retrieval Techniques参考文献 21被引用 10

一句话总结

本文提出一种基于动态相似性匹配（DSM）网络的联合定位与姿态估计方法，用于跨视角地理定位。通过将航拍图像应用极坐标变换以减小域差距，并在不同方位角上关联地面图像与变换后的航拍图像特征，该方法在CVUSA数据集上将未知姿态的180°视场角图像的top-1定位召回率提升了高达6倍。

ABSTRACT

Cross-view geo-localization is the problem of estimating the position and orientation (latitude, longitude and azimuth angle) of a camera at ground level given a large-scale database of geo-tagged aerial (e.g., satellite) images. Existing approaches treat the task as a pure location estimation problem by learning discriminative feature descriptors, but neglect orientation alignment. It is well-recognized that knowing the orientation between ground and aerial images can significantly reduce matching ambiguity between these two views, especially when the ground-level images have a limited Field of View (FoV) instead of a full field-of-view panorama. Therefore, we design a Dynamic Similarity Matching network to estimate cross-view orientation alignment during localization. In particular, we address the cross-view domain gap by applying a polar transform to the aerial images to approximately align the images up to an unknown azimuth angle. Then, a two-stream convolutional network is used to learn deep features from the ground and polar-transformed aerial images. Finally, we obtain the orientation by computing the correlation between cross-view features, which also provides a more accurate measure of feature similarity, improving location recall. Experiments on standard datasets demonstrate that our method significantly improves state-of-the-art performance. Remarkably, we improve the top-1 location recall rate on the CVUSA dataset by a factor of 1.5x for panoramas with known orientation, by a factor of 3.3x for panoramas with unknown orientation, and by a factor of 6x for 180-degree FoV images with unknown orientation.

研究动机与目标

• 解决在位置与姿态均未知的情况下进行跨视角地理定位的挑战。
• 减小由于视角差异极大而导致的地面图像与航拍图像之间的巨大域差距。
• 缓解因地面图像视场角受限且姿态未知而引起的定位模糊性。
• 通过跨视角相关性联合估计姿态与位置，提升特征匹配的准确性。
• 实现在姿态信息无法预先获取的真实世界场景下的鲁棒定位。

提出的方法

• 对航拍图像应用极坐标变换，使其近似对齐地面视角几何结构，从而减小域差距。
• 采用双流卷积神经网络分别从地面图像与极坐标变换后的航拍图像中提取深层特征。
• 实现一个动态相似性匹配（DSM）模块，沿方位角方向计算跨视角特征的相关性，以估计姿态。
• 根据估计的姿态对航拍图像特征进行平移与裁剪，以提升特征相似性与定位检索精度。
• 将相关性峰值作为姿态估计结果，并计算特征距离以实现定位匹配。
• 利用空间感知的特征体积，以保留对定位至关重要的几何关系。

实验结果

研究问题

• RQ1 当姿态未知时，联合估计位置与姿态是否能提升跨视角地理定位性能？
• RQ2 极坐标变换在减小地面视图与航拍视图之间域差距方面效果如何？
• RQ3 在视场角受限的场景下，姿态感知的特征匹配能在多大程度上减少定位模糊性？
• RQ4 基于方位角的相关性方法能否准确估计地面图像的姿态？
• RQ5 在不同视场角与姿态条件下，该方法与最先进方法相比在top-1召回率上的表现如何？

主要发现

• 对于姿态已知的全景图像，该方法在CVUSA数据集上的top-1定位召回率提升了1.5倍。
• 对于姿态未知的全景图像，top-1召回率相比最先进方法提升了3.3倍。
• 对于姿态未知的180°视场角图像，top-1召回率的提升达到6倍。
• 在CVUSA数据集上，360°视场角图像的姿态预测准确率为99.41%，70°视场角图像为61.67%。
• 所有视场角设置下的中位数姿态误差均低于5°，表明姿态估计具有高精度。
• 定性结果表明，即使在小视场角图像上，也能实现准确的联合定位与姿态估计，且相关性峰值与真实值高度对齐。