[Paper Review] InLoc: Indoor Visual Localization with Dense Matching and View Synthesis

Hajime Taira, Masatoshi Okutomi+6 more|arXiv (Cornell University)|Mar 28, 2018

Robotics and Sensor-Based Localization2 references46 citations

TL;DR

InLoc proposes a large-scale indoor visual localization pipeline that uses dense CNN-based matching and virtual view synthesis for robust 6DoF pose estimation against challenging indoor changes. It introduces a new dataset and shows significant gains over state-of-the-art methods.

ABSTRACT

We seek to predict the 6 degree-of-freedom (6DoF) pose of a query photograph with respect to a large indoor 3D map. The contributions of this work are three-fold. First, we develop a new large-scale visual localization method targeted for indoor environments. The method proceeds along three steps: (i) efficient retrieval of candidate poses that ensures scalability to large-scale environments, (ii) pose estimation using dense matching rather than local features to deal with textureless indoor scenes, and (iii) pose verification by virtual view synthesis to cope with significant changes in viewpoint, scene layout, and occluders. Second, we collect a new dataset with reference 6DoF poses for large-scale indoor localization. Query photographs are captured by mobile phones at a different time than the reference 3D map, thus presenting a realistic indoor localization scenario. Third, we demonstrate that our method significantly outperforms current state-of-the-art indoor localization approaches on this new challenging data.

Motivation & Objective

• Address the challenge of accurate 6DoF localization in large-scale indoor environments with textureless and repetitive structures.
• Propose a localization pipeline that combines efficient candidate pose retrieval, dense feature-based pose estimation, and verification via view synthesis.
• Create and release a realistic indoor localization dataset with smartphone-query images and temporally offset references.
• Demonstrate that dense matching and view-synthesis verification outperform existing indoor localization approaches.

Proposed method

• Perform coarse-to-fine dense matching for pose estimation using CNN-derived features (conv5 then conv3) on a grid to overcome lack of sparse features.
• Retrieve candidate database images with NetVLAD descriptors and re-rank by RANSAC inliers to reduce candidates (top-10).
• Estimate 6DoF pose with P3P-LO-RANSAC using depth-backed 3D structure from the database image.
• Verify poses by virtual view synthesis: render a view from the estimated pose and compare with the query using DenseSIFT/RootSIFT descriptors to measure both positive and negative evidence across the image, robust to illumination changes.
• Use binarization of CNN features to reduce memory with negligible loss, enabling scalable matching.
• Evaluate and compare against baselines (Direct 2D-3D, Disloc, NetVLAD with SparsePE) and ablate components (DensePE, DensePV).

Experimental results

Research questions

• RQ1 How can dense feature matching improve pose estimation in textureless indoor scenes compared to sparse feature methods?
• RQ2 What is the effect of pose verification via view synthesis on localization accuracy in indoor environments with large viewpoint and illumination changes?
• RQ3 Does combining dense matching with robust retrieval and verification provide measurable gains over state-of-the-art indoor localization methods on large-scale indoor data?

Key findings

• InLoc outperforms state-of-the-art indoor localization baselines on the new large-scale dataset, with substantial gains in correctly localized queries at 0.25m, 0.50m, and 1.00m thresholds.
• Dense pose estimation (DensePE) improves localization rate by ~15 percentage points over sparse feature matching when using NetVLAD-based retrieval.
• Pose verification with view synthesis (DensePV) provides significant and consistent improvements, especially for position accuracy within 1.5m.
• Binary CNN features reduce memory by 32x with negligible performance loss (<1% at 0.5m).
• Compared to Disloc and NetVLAD baselines, InLoc achieves notably higher localization accuracy (e.g., 0.25m: 38.9% vs 11.9%/21.3%), demonstrating the benefit of dense matching and view synthesis.