[Paper Review] Enabling Socially Competent navigation through incorporating HRI

Arturo Cruz-Maya, Fernando García+1 more|arXiv (Cornell University)|Apr 19, 2019

Video Surveillance and Tracking Methods18 references4 citations

TL;DR

This paper presents a novel framework that integrates human-robot interaction (HRI) into robot navigation to enable socially competent behavior when paths are blocked by people. By combining ROS navigation, OpenCV, and MobileNet SSD for real-time people detection, the system pauses navigation, assesses social context, and initiates verbal permission requests before resuming, successfully enabling natural human-robot collaboration in real-world indoor environments.

ABSTRACT

Over the last years, social robots have been deployed in public environments making evident the need of human-aware navigation capabilities. In this regard, the robotics community have made efforts to include proxemics or social conventions within the navigation approaches. Nevertheless, few works have tackled the problem of labelling humans as an interactive agent when blocking the robot motion trajectory. Current state of the art navigation planners will either propose an alternative path or freeze the motion until the path is free. We present the first prototype of a framework designed to enhance social competency of robots while navigating in indoor environments. The implementation is done using Navigation and Object Detection open-source software. Specifically, the Robot Operating System (ROS) navigation stack, and OpenCV with Caffe deep learning models and MobileNet Single Shot Detector (SSD), respectively.

Motivation & Objective

• Address the lack of socially aware navigation in dynamic public environments where robots must interact with people blocking their path.
• Overcome limitations of current navigation planners that either freeze or reroute without human interaction when blocked.
• Develop and deploy a functional system that treats people as interactive agents, enabling dialog-based permission requests during navigation.
• Enable real-world deployment of socially competent navigation on a humanoid robot (Pepper) using open-source software stacks.
• Introduce a situation assessment module that triggers HRI only when necessary, based on path length and obstacle proximity metrics.

Proposed method

• Uses the ROS navigation stack for global and local path planning with costmaps and dynamic reconfiguration.
• Employs OpenCV with Caffe and MobileNet Single Shot Detector (SSD) for real-time person detection from depth and RGB data.
• Applies MeanShift clustering to laser scan data to detect human clusters when path is blocked (distance < 3m).
• Triggers a situation assessment module when path is obstructed, comparing new path length to original to determine if interaction is needed.
• Stops navigation and initiates a verbal interaction via NAOqi application if path is blocked and new path is longer than original.
• Uses a 5-second timeout to re-prompt for permission if the path remains blocked, ensuring persistence without freezing.

Experimental results

Research questions

• RQ1 How can robots navigate in human-populated environments while respecting social norms when blocked by people?
• RQ2 Can a robot’s navigation system be enhanced with HRI to request permission to pass, rather than rerouting or freezing?
• RQ3 What conditions trigger a socially appropriate interaction instead of automatic path re-planning?
• RQ4 How does path length comparison between original and new trajectories influence the decision to interact or reroute?
• RQ5 Can a real-world deployment of such a system demonstrate successful human-robot collaboration in unplanned, dynamic scenarios?

Key findings

• The system successfully detects blocked paths using laser scan clustering and person detection, triggering interaction only when necessary.
• In real-world trials, the robot paused navigation and initiated a verbal permission request when blocked, resulting in natural human-robot collaboration.
• One unexpected outcome was that a person overheard the robot’s request and proactively asked the other person to clear the path, demonstrating emergent cooperation.
• The path length comparison mechanism allowed the robot to avoid interaction when a shorter path was available, preserving efficiency.
• The system demonstrated robustness in dynamic indoor environments such as hallways with multiple people, maintaining social competence.
• Despite performance penalties from offboard inference, the framework was successfully deployed and tested on a humanoid robot (Pepper) in real-world scenarios.