[Paper Review] Large-scale Image Geo-Localization Using Dominant Sets

Eyasu Zemene, Yonatan Tariku+4 more|arXiv (Cornell University)|Feb 4, 2017

Advanced Image and Video Retrieval Techniques4 references4 citations

TL;DR

This paper proposes a fast and accurate large-scale image geo-localization framework using Dominant Set Clustering (DSC) for robust feature matching and constrained clustering. By dynamically selecting multiple nearest neighbors based on feature discrimination and leveraging efficient game-theoretic dynamics, the method achieves 20% and 7% accuracy gains over state-of-the-art on 102k and 300k image datasets, respectively, while being 200x faster than prior methods.

ABSTRACT

This paper presents a new approach for the challenging problem of geo-locating an image using image matching in a structured database of city-wide reference images with known GPS coordinates. We cast the geo-localization as a clustering problem on local image features. Akin to existing approaches on the problem, our framework builds on low-level features which allow partial matching between images. For each local feature in the query image, we find its approximate nearest neighbors in the reference set. Next, we cluster the features from reference images using Dominant Set clustering, which affords several advantages over existing approaches. First, it permits variable number of nodes in the cluster which we use to dynamically select the number of nearest neighbors (typically coming from multiple reference images) for each query feature based on its discrimination value. Second, as we also quantify in our experiments, this approach is several orders of magnitude faster than existing approaches. Thus, we obtain multiple clusters (different local maximizers) and obtain a robust final solution to the problem using multiple weak solutions through constrained Dominant Set clustering on global image features, where we enforce the constraint that the query image must be included in the cluster. This second level of clustering also bypasses heuristic approaches to voting and selecting the reference image that matches to the query. We evaluated the proposed framework on an existing dataset of 102k street view images as well as a new dataset of 300k images, and show that it outperforms the state-of-the-art by 20% and 7%, respectively, on the two datasets.

Motivation & Objective

• Address the challenge of accurate and efficient large-scale image geo-localization using visual features.
• Overcome limitations of existing methods that rely on fixed nearest neighbor counts and heuristic voting schemes.
• Develop a scalable, fast, and robust solution for city-scale image matching with known GPS coordinates.
• Enable dynamic selection of nearest neighbors per query feature based on discriminative power to improve matching accuracy.
• Introduce a constrained DSC framework that ensures the query image is included in the final cluster, bypassing heuristic voting.

Proposed method

• Use Dominant Set Clustering (DSC) to group local features from reference images, enabling variable cluster sizes and robust outlier handling.
• Apply infection-immunization dynamics (InImDyn) to solve the DSC problem with linear time/space complexity, enabling speedups of up to 200x.
• Dynamically select multiple nearest neighbors per query feature based on feature discrimination—more neighbors for ambiguous features, fewer for clear ones.
• Use a query-constrained DSC formulation to combine multiple local maxima into a single robust solution, ensuring the query is included in the final cluster.
• Fusion of global features (e.g., NetVLAD, GIST, HSV) is weighted based on normalized similarity scores to build a discriminative input graph for post-processing.
• Integrate the DSC post-processing step with any retrieval pipeline, adding less than 0.003s per query on average.

Experimental results

Research questions

• RQ1 Can adaptive nearest neighbor selection based on feature discrimination improve geo-localization accuracy?
• RQ2 Can Dominant Set Clustering with game-theoretic dynamics outperform NP-hard formulations like GMCP in speed and accuracy?
• RQ3 Does constrained DSC that includes the query image in the cluster eliminate the need for heuristic voting schemes?
• RQ4 To what extent does fusing multiple global features enhance the performance of the post-processing step?
• RQ5 Can the proposed method scale effectively to large datasets (e.g., 300k images) while maintaining high accuracy and low latency?

Key findings

• The proposed method achieves a 20% accuracy improvement over state-of-the-art on the 102k Google Street View dataset.
• On the larger 300k-image WorldCities dataset, the method improves accuracy by 7% compared to prior methods.
• The framework is on average 200 times faster than the GMCP-based approach due to efficient InImDyn dynamics and linear relaxation.
• Post-processing with constrained DSC boosts rank-1 accuracy by up to 7% when applied to existing retrieval pipelines, with less than 0.003s additional latency per query.
• Fusing multiple global features (NetVLAD, CNN6, CNN7, GIST, HSV) based on their discriminative power yields better performance than using individual features.
• The constrained DSC approach successfully bypasses heuristic voting by directly enforcing query inclusion in the final cluster, leading to more robust and consistent results.