[Paper Review] Towards an immersive user interface for waypoint navigation of a mobile robot

Gregory S. Baker, Tom Bridgwater+2 more|arXiv (Cornell University)|Mar 28, 2020

Virtual Reality Applications and Impacts11 citations

TL;DR

This paper proposes an immersive head-mounted display (HMD)-based waypoint navigation interface for mobile robots using 6-DOF tracked controllers to point at target locations in a virtual environment. A user study with 37 participants showed the method enabled quick learning, reduced workload, and resilience to communication delay, though VR sickness remained a significant challenge in egocentric HMD setups.

ABSTRACT

In this paper, we investigate the utility of head-mounted display (HMD) interfaces for navigation of mobile robots. We focus on the selection of waypoint positions for the robot, whilst maintaining an egocentric view of the robot's environment. Inspired by virtual reality (VR) gaming, we propose a target selection method that uses the 6 degrees-of-freedom tracked controllers of a commercial VR headset. This allows an operator to point to the desired target position, in the vicinity of the robot, which the robot then autonomously navigates towards. A user study (37 participants) was conducted to examine the efficacy of this control strategy when compared to direct control, both with and without a communication delay. The results of the experiment showed that participants were able to learn how to use the novel system quickly, and the majority of participants reported a preference for waypoint control. Across all recorded metrics (task performance, operator workload and usability) the proposed waypoint control interface was not significantly affected by the communication delay, in contrast to direct control. The simulated experiment indicated that a real-world implementation of the proposed interface could be effective, but also highlighted the need to manage the negative effects of HMDs - particularly VR sickness.

Motivation & Objective

• To investigate the effectiveness of immersive HMD interfaces for waypoint navigation in mobile robot teleoperation.
• To design and evaluate a novel target-selection method using 6-DOF tracked controllers inspired by VR gaming.
• To compare waypoint control against direct control under varying communication delay conditions.
• To assess the impact of egocentric HMD viewing on user workload, task performance, and VR sickness.
• To identify practical challenges and limitations for real-world deployment of HMD-based teleoperation systems.

Proposed method

• The system uses an HTC Vive HMD with tracked motion controllers to enable point-and-select waypoint navigation in a virtual replica of the robot’s environment.
• Target selection is implemented using a VR gaming-inspired teleportation method, where users point at a location and confirm via controller input.
• The interface provides an egocentric viewpoint, simulating the robot’s first-person perspective through its onboard camera feed.
• A virtual reality user study was conducted with 37 participants, comparing waypoint control to direct control under 0s and 1s communication delays.
• Task performance, operator workload (NASA-TLX), usability (SUS), and post-experiment interviews were used to evaluate the interface.
• VR sickness was monitored through participant self-reports, with nausea severity noted as a key side effect.

Experimental results

Research questions

• RQ1 Does an HMD-based waypoint navigation interface with controller-based target selection improve task performance and usability compared to direct control?
• RQ2 How does communication delay (0s vs. 1s) affect performance and workload under waypoint vs. direct control?
• RQ3 To what extent does the egocentric HMD viewpoint contribute to VR sickness during robot teleoperation?
• RQ4 How quickly can users learn to use the proposed waypoint interface in a simulated environment?
• RQ5 What are the subjective and objective differences in workload and usability between the two control strategies?

Key findings

• Participants learned to use the waypoint interface quickly, with no significant learning curve observed across conditions.
• Waypoint control significantly reduced operator workload compared to direct control, especially under 1-second delay.
• The waypoint interface was not significantly affected by communication delay, whereas direct control performance declined under delay.
• A majority of participants preferred the waypoint control interface due to reduced cognitive load and improved task efficiency.
• VR sickness was a major issue, with 3 participants unable to complete all conditions and many reporting mild to moderate nausea, particularly in the delayed direct control condition.
• The study suggests that while the interface is effective in simulation, real-world implementation must address VR sickness and environmental mapping fidelity.