[Paper Review] Object Goal Navigation using Goal-Oriented Semantic Exploration

Devendra Singh Chaplot, Dhiraj Gandhi+2 more|arXiv (Cornell University)|Jul 1, 2020

Multimodal Machine Learning Applications50 references221 citations

TL;DR

SemExp builds a semantic map and a goal-driven exploration policy to efficiently navigate to object categories in unseen environments, outperforming baselines and winning the CVPR 2020 Habitat ObjectNav Challenge. It transfers well to real robots.

ABSTRACT

This work studies the problem of object goal navigation which involves navigating to an instance of the given object category in unseen environments. End-to-end learning-based navigation methods struggle at this task as they are ineffective at exploration and long-term planning. We propose a modular system called, `Goal-Oriented Semantic Exploration' which builds an episodic semantic map and uses it to explore the environment efficiently based on the goal object category. Empirical results in visually realistic simulation environments show that the proposed model outperforms a wide range of baselines including end-to-end learning-based methods as well as modular map-based methods and led to the winning entry of the CVPR-2020 Habitat ObjectNav Challenge. Ablation analysis indicates that the proposed model learns semantic priors of the relative arrangement of objects in a scene, and uses them to explore efficiently. Domain-agnostic module design allow us to transfer our model to a mobile robot platform and achieve similar performance for object goal navigation in the real-world.

Motivation & Objective

• Motivate and address object goal navigation in unseen environments by leveraging semantics in explicit episodic memory.
• Propose a modular system that combines semantic mapping with a goal-oriented exploration policy to improve exploration efficiency and long-term planning.
• Leverage pretrained first-person semantic predictions with differentiable projections to build top-down semantic maps.
• Demonstrate state-of-the-art performance in simulated habitats and transferability to real-world robotic platforms.

Proposed method

• Introduce SemExp with two learnable modules: Semantic Mapping and Goal-Oriented Semantic Policy.
• Build a top-down semantic map with obstacle/explored channels plus 15 object-category channels; use differentiable projection from first-person predictions.
• Predict semantic categories from RGB with pretrained detectors (Mask R-CNN) and project to top-down space; apply a denoising network to obtain the map.
• Train the Goal-Oriented Semantic Policy to select long-term goals based on semantic priors; use PPO with a reward based on distance reduction to the nearest goal object.
• Use a deterministic local planner (Fast Marching Method) to reach the long-term goal from the current location, updating the map and replanning each step.

Experimental results

Research questions

• RQ1 How can explicit semantic maps improve object-goal navigation in unseen environments?
• RQ2 Does a goal-oriented semantic exploration policy outperform goal-agnostic exploration in navigation tasks?
• RQ3 Can semantic priors about object arrangement in scenes be learned and exploited to enhance exploration efficiency?
• RQ4 How well does the SemExp framework transfer from simulation to real-world robotic platforms?

Key findings

• SemExp outperforms all baselines on Gibson and MP3D datasets, with higher SPL and Success and lower DTS compared to Active Neural SLAM.
• Ablations show that both semantic mapping and the goal-driven policy contribute significantly to performance; GT semantic segmentation further improves results.
• Using ground-truth semantic segmentation yields a substantial increase in success (73.1% vs 54.4%), highlighting semantic prediction quality as a key bottleneck.
• SemExp achieved 25.3% success in the CVPR 2020 Habitat ObjectNav Challenge, outperforming the next best entry.
• Real-world transfer experiments with Locobot show 65% success, indicating domain-agnostic design and reliance on pretrained detectors enable effective sim-to-real transfer.