[Paper Review] Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation

Haresh Karnan, Anirudh Nair+7 more|arXiv (Cornell University)|Mar 28, 2022

Social Robot Interaction and HRI4 citations

TL;DR

This paper introduces SCAND, a large-scale, first-person-view dataset of 8.7 hours and 138 socially compliant navigation trajectories collected via human teleoperation on two distinct robots (Boston Dynamics Spot and Clearpath Jackal) in diverse indoor and outdoor environments. The dataset includes multi-modal data (3D LiDAR, joystick commands, odometry, visual, and IMU) and labeled social interaction annotations, enabling imitation learning to train socially compliant global and local navigation policies that outperform baseline methods in human evaluations.

ABSTRACT

Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a 'socially compliant' manner in the presence of other intelligent agents such as humans. With the emergence of autonomously navigating mobile robots in human populated environments (e.g., domestic service robots in homes and restaurants and food delivery robots on public sidewalks), incorporating socially compliant navigation behaviors on these robots becomes critical to ensuring safe and comfortable human robot coexistence. To address this challenge, imitation learning is a promising framework, since it is easier for humans to demonstrate the task of social navigation rather than to formulate reward functions that accurately capture the complex multi objective setting of social navigation. The use of imitation learning and inverse reinforcement learning to social navigation for mobile robots, however, is currently hindered by a lack of large scale datasets that capture socially compliant robot navigation demonstrations in the wild. To fill this gap, we introduce Socially CompliAnt Navigation Dataset (SCAND) a large scale, first person view dataset of socially compliant navigation demonstrations. Our dataset contains 8.7 hours, 138 trajectories, 25 miles of socially compliant, human teleoperated driving demonstrations that comprises multi modal data streams including 3D lidar, joystick commands, odometry, visual and inertial information, collected on two morphologically different mobile robots a Boston Dynamics Spot and a Clearpath Jackal by four different human demonstrators in both indoor and outdoor environments. We additionally perform preliminary analysis and validation through real world robot experiments and show that navigation policies learned by imitation learning on SCAND generate socially compliant behaviors

Motivation & Objective

• To address the lack of large-scale, real-world datasets capturing socially compliant robot navigation in diverse environments.
• To enable imitation learning for mobile robot navigation by providing high-fidelity human demonstration data.
• To support the development and evaluation of socially compliant navigation policies through real-world validation.
• To provide a benchmark for studying social navigation strategies, trajectory classification, and inverse reinforcement learning.
• To facilitate research in representation learning, real-to-sim transfer, and human-robot interaction modeling.

Proposed method

• Collecting 138 trajectories across 25 miles using two morphologically different robots: Boston Dynamics Spot and Clearpath Jackal.
• Recording multi-modal data streams: 3D LiDAR, joystick commands, odometry, RGB camera, and 6D IMU from human teleoperation.
• Labeling each trajectory with natural social interactions such as yielding, following traffic rules, and navigating around crowds.
• Training a behavior cloning (BC) agent on the demonstration data to learn both global and local navigation policies.
• Evaluating the learned policy through human-in-the-loop trials comparing social compliance and safety against a baseline move_base controller.
• Performing classifier-based analysis to demonstrate that distinct human demonstrators use different socially compliant navigation strategies.

Experimental results

Research questions

• RQ1 Can a large-scale, real-world dataset of human-teleoperated robot navigation capture diverse, socially compliant behaviors in unstructured environments?
• RQ2 To what extent do different human demonstrators exhibit distinct strategies for socially compliant navigation, and can these be learned from demonstration?
• RQ3 Can imitation learning on SCAND successfully train navigation policies that are perceived as socially compliant and safe by human evaluators?
• RQ4 How effective is behavior cloning on SCAND for learning both global and local navigation policies in complex, dynamic environments?
• RQ5 Can SCAND be used to train models for downstream tasks such as trajectory prediction, classification, and inverse reinforcement learning?

Key findings

• A neural network classifier achieved 74.48% accuracy in distinguishing between navigation strategies of two different human demonstrators, demonstrating that distinct socially compliant strategies exist in the data.
• Behavior cloning on SCAND successfully learned a socially compliant local navigation policy that outperformed a naive move_base baseline in human evaluation.
• Human participants rated the behavior-cloned agent as significantly more socially compliant and safer than the baseline agent in two real-world navigation scenarios.
• The dataset contains 8.7 hours of demonstrations, 138 trajectories, and 25 miles of navigation across indoor and outdoor environments, with rich multi-modal sensor data and social interaction annotations.
• The dataset supports diverse research applications, including imitation learning, trajectory classification, inverse reinforcement learning, and real-to-sim transfer.
• SCAND enables the study of regional navigation norms (e.g., driving on the right) and supports the development of generalizable, socially aware robot navigation policies.