[Paper Review] High-Level Plan for Behavioral Robot Navigation with Natural Language Directions and R-NET

Amar Shrestha, Krittaphat Pugdeethosapol+2 more|arXiv (Cornell University)|Jan 8, 2020

Multimodal Machine Learning Applications10 references4 citations

TL;DR

This paper proposes a novel R-NET-based model with gated and self-matching attention to translate natural language navigation instructions into high-level behavioral plans for robots in known environments. By modeling the navigational space as a graph of landmarks and behaviors, the method uses a pointer network to generate optimal sequential behavior plans, outperforming state-of-the-art approaches on both known and unknown environment benchmarks.

ABSTRACT

When the navigational environment is known, it can be represented as a graph where landmarks are nodes, the robot behaviors that move from node to node are edges, and the route is a set of behavioral instructions. The route path from source to destination can be viewed as a class of combinatorial optimization problems where the path is a sequential subset from a set of discrete items. The pointer network is an attention-based recurrent network that is suitable for such a task. In this paper, we utilize a modified R-NET with gated attention and self-matching attention translating natural language instructions to a high-level plan for behavioral robot navigation by developing an understanding of the behavioral navigational graph to enable the pointer network to produce a sequence of behaviors representing the path. Tests on the navigation graph dataset show that our model outperforms the state-of-the-art approach for both known and unknown environments.

Motivation & Objective

• To enable robots to interpret natural language directions into actionable behavioral navigation plans.
• To address the challenge of mapping natural language instructions to a sequence of discrete robot behaviors in a structured navigational graph.
• To improve path planning performance in both known and unknown environments using neural sequence generation.
• To develop a pointer network-based approach that leverages attention mechanisms for combinatorial optimization of navigational sequences.

Proposed method

• The navigational environment is modeled as a graph where landmarks are nodes and robot behaviors are edges.
• A modified R-NET with gated attention and self-matching attention is used to encode natural language instructions and predict a sequence of behaviors.
• The pointer network generates a path as a sequence of behaviors by selecting from a discrete set of actions in the navigational graph.
• The model is trained end-to-end to map input language descriptions to high-level behavioral plans using supervised learning.
• Attention mechanisms allow the model to focus on relevant parts of the instruction and the graph structure during sequence generation.
• The approach treats path planning as a combinatorial optimization problem over discrete behavioral actions.

Experimental results

Research questions

• RQ1 Can a neural network effectively map natural language navigation instructions to a sequence of robot behaviors in a known environment?
• RQ2 How well does the proposed R-NET model generalize to unknown environments compared to existing methods?
• RQ3 To what extent do gated attention and self-matching attention improve the model's ability to understand and generate navigational plans?
• RQ4 Can the pointer network effectively solve the combinatorial optimization problem of selecting the optimal path from a set of discrete behaviors?

Key findings

• The proposed model outperforms the state-of-the-art approach on the navigation graph dataset for both known and unknown environments.
• The integration of gated attention and self-matching attention enhances the model's understanding of complex natural language instructions.
• The pointer network successfully generates valid and optimal behavioral sequences for robot navigation.
• The model demonstrates robust generalization to unseen environments, indicating strong generalization capability.