[Paper Review] Condition-Invariant Multi-View Place Recognition

José M. Fácil, Daniel Olid+2 more|arXiv (Cornell University)|Feb 25, 2019

Robotics and Sensor-Based Localization36 references39 citations

TL;DR

The paper proposes three deep-learning based multi-view architectures (Descriptor Grouping, Descriptor Fusion, and Recurrent Descriptors) to improve visual place recognition across appearance changes by learning from sequences of frames. These models outperform single-view and SeqSLAM baselines while using significantly smaller descriptors.

ABSTRACT

Visual place recognition is particularly challenging when places suffer changes in its appearance. Such changes are indeed common, e.g., due to weather, night/day or seasons. In this paper we leverage on recent research using deep networks, and explore how they can be improved by exploiting the temporal sequence information. Specifically, we propose 3 different alternatives (Descriptor Grouping, Fusion and Recurrent Descriptors) for deep networks to use several frames of a sequence. We show that our approaches produce more compact and best performing descriptors than single- and multi-view baselines in the literature in two public databases.

Motivation & Objective

• Address robustness of visual place recognition under appearance changes (weather, day/night, seasons, dynamics).
• Leverage multi-view information by learning from frame sequences instead of relying on single images.
• Develop and evaluate three deep learning architectures to fuse multi-frame descriptors efficiently.

Proposed method

• Use ResNet-50 as backbone to extract 128-dimensional descriptors per frame.
• Descriptor Grouping: concatenate descriptors from a window of n frames to form a 128×n sequence descriptor.
• Descriptor Fusion: learn a 128-dimensional fused descriptor from the outputs of n frames via an extra fully connected layer.
• Recurrent Descriptors: feed per-frame descriptors into an LSTM to produce a 128-dimensional sequence descriptor that updates over time.
• Train all models with a triplet loss (Wohlhart-Lepetit) to pull same-place sequences together and push different-place sequences apart.
• Evaluate on Nordland and Alderley datasets and compare to single-view baselines and SeqSLAM.

Experimental results

Research questions

• RQ1 Can multi-view deep representations improve place recognition under severe appearance changes?
• RQ2 Which multi-view fusion strategy (grouping, fusion, or recurrence) yields the best trade-off between accuracy and descriptor compactness?
• RQ3 How do these models perform under variations such as reverse sequence order or speed changes?

Key findings

• Descriptor Grouping achieved the highest precision with 3 frames on Partitioned Nordland (92% precision) using a 384-element descriptor.
• Descriptor Fusion achieved 87% precision with 3 frames on Nordland and outperformed many baselines with a smaller descriptor size (128).
• Recurrent Descriptors achieved 85% precision with 3 frames on Nordland and showed strong robustness to sequence dynamics.
• All three multi-view models outperformed the single-view baseline and SeqSLAM when using the same number of frames.
• Descriptor sizes are substantially smaller (e.g., 384 or 128 dimensions) than many baselines while delivering higher accuracy.
• On Alderley (day vs night), the multi-view approach (grouping) remained the best performing model.