[Paper Review] ScoutBot: A Dialogue System for Collaborative Navigation

Stephanie M. Lukin, Felix Gervits+8 more|arXiv (Cornell University)|Jul 21, 2018

Speech and dialogue systems9 references4 citations

TL;DR

ScoutBot is a dialogue system that enables natural language control of a simulated robot for collaborative navigation in urban environments. It uses a trained natural language understanding (NLU) pipeline to interpret unconstrained spoken commands, resolve ambiguities via clarification dialogues, and execute actions through ROS-based motion commands, demonstrating robust, low-bandwidth human-robot interaction in simulation.

ABSTRACT

ScoutBot is a dialogue interface to physical and simulated robots that supports collaborative exploration of environments. The demonstration will allow users to issue unconstrained spoken language commands to ScoutBot. ScoutBot will prompt for clarification if the user's instruction needs additional input. It is trained on human-robot dialogue collected from Wizard-of-Oz experiments, where robot responses were initiated by a human wizard in previous interactions. The demonstration will show a simulated ground robot (Clearpath Jackal) in a simulated environment supported by ROS (Robot Operating System).

Motivation & Objective

• To develop a data-driven, automated dialogue system that supports natural language interaction with a robot for collaborative exploration.
• To reduce human cognitive load and communication bandwidth in remote or high-stress environments like disaster response by enabling high-level, spoken commands.
• To simulate and validate human-robot interaction using a Wizard-of-Oz framework before transitioning to full automation.
• To enable clarification dialogues for ambiguous or incomplete instructions, improving robustness in real-world deployment.
• To create a scalable research platform for testing NLU, dialogue policies, and navigation strategies in simulated and real environments.

Proposed method

• The system uses a simulated Clearpath Jackal robot in a Gazebo environment with ROS for sensor and motion control.
• Natural language input is processed by an NLU classifier trained on Wizard-of-Oz dialogue data to extract action commands.
• Commands are mapped to ROS Twist messages for motion control, supporting movements like forward/backward (1–10 ft) and turns (45°–360°).
• Clarification subdialogues are triggered when commands are ambiguous or incomplete, mimicking human-like interaction.
• The system uses rviz for real-time visualization of sensor data and robot state, with identical configuration for physical and simulated robots.
• The dialogue manager and robot navigator roles are automated using Python scripts and the rospy ROS package.

Experimental results

Research questions

• RQ1 How can a dialogue system interpret unconstrained spoken language commands to control a robot’s navigation in a complex environment?
• RQ2 What types of clarification strategies are effective in resolving ambiguous or underspecified natural language instructions during human-robot collaboration?
• RQ3 How does the integration of NLU and ROS-based control enable robust, low-bandwidth interaction in simulated environments?
• RQ4 What are the key components of a scalable architecture for transitioning from Wizard-of-Oz experiments to automated dialogue systems?
• RQ5 How can context-aware language understanding (e.g., relative landmarks) be integrated into a dialogue-driven navigation system?

Key findings

• ScoutBot successfully interprets a variety of unconstrained natural language commands, including metric-based navigation instructions such as 'turn left 90 degrees' or 'move forward 5 feet'.
• The system effectively triggers clarification dialogues when commands are ambiguous, preventing incorrect or unsafe robot actions.
• The simulated environment with ROS and Gazebo enables safe, repeatable testing of dialogue and navigation systems prior to real-world deployment.
• The same rviz visualization configuration works for both simulated and physical robots, ensuring consistency across test environments.
• The automated dialogue and navigation pipeline demonstrates feasibility for transitioning from Wizard-of-Oz studies to fully autonomous human-robot collaboration.
• The system currently does not support landmark-based navigation or collision avoidance, indicating a key area for future extension.