[Paper Review] Unifying Map and Landmark Based Representations for Visual Navigation

Saurabh Gupta, David F. Fouhey+2 more|arXiv (Cornell University)|Dec 21, 2017

Robotics and Sensor-Based Localization32 references64 citations

TL;DR

The paper presents a differentiable, end-to-end learned framework that combines map-based planning with landmark-based execution using path signatures and feature synthesis to robustly navigate with sparse observations and noisy actuation.

ABSTRACT

This works presents a formulation for visual navigation that unifies map based spatial reasoning and path planning, with landmark based robust plan execution in noisy environments. Our proposed formulation is learned from data and is thus able to leverage statistical regularities of the world. This allows it to efficiently navigate in novel environments given only a sparse set of registered images as input for building representations for space. Our formulation is based on three key ideas: a learned path planner that outputs path plans to reach the goal, a feature synthesis engine that predicts features for locations along the planned path, and a learned goal-driven closed loop controller that can follow plans given these synthesized features. We test our approach for goal-driven navigation in simulated real world environments and report performance gains over competitive baseline approaches.

Motivation & Objective

• Motivate navigation with both map-based planning and landmark-based execution to handle noisy actuation.
• Propose a learnable pipeline that builds space representations from sparse registered views.
• Develop a path planner that outputs paths and a feature synthesizer to produce landmarks along the path.
• Introduce a closed-loop controller that follows plans using synthesized features to compensate for drift.

Proposed method

• Map generation: transform sparse registered images into an allocentric spatial map using an egocentric-to-allocentric transformation and dense fusion.
• Path planning: learn a value-iteration-based planner over the learned map to produce a path and corresponding action sequence.
• Path signatures: synthesize features for locations along the path from neighboring views to create a robust path descriptor.
• Plan execution: train a recurrent policy that consumes path signatures and current observation to execute the plan under actuation noise.
• Feature synthesis: fuse representations from multiple reference images weighted by relative pose to approximate features at unseen locations.

Experimental results

Research questions

• RQ1 Can a learned map from sparse views support reliable path planning and shortcut discovery?
• RQ2 How well can the system synthesize features for unseen locations to support localization and drift correction?
• RQ3 Does incorporating path signatures improve robustness of plan execution under actuation noise?
• RQ4 How does the joint mapper-planner and path-signature execution compare to baselines with open-loop or non-synthesized features?

Key findings

• Joint mapper and planner can generate meaningful paths from sparse environmental views.
• Feature synthesis outperforms traditional SIFT-based baselines for predicting location-sensitive features.
• Path signatures enable robust plan execution under noise without explicit relocalization or replanning.
• Learned architectures for memory and control outperform standard memory-based networks on navigation tasks in simulated real-world office reconstructions.