[Paper Review] Navigation by Imitation in a Pedestrian-Rich Environment

Jing Bi, Tianyou Xiao+2 more|arXiv (Cornell University)|Nov 1, 2018

Robotic Path Planning Algorithms9 references9 citations

TL;DR

This paper proposes a Learn-from-Intervention DAgger algorithm that enables a robot to navigate autonomously in pedestrian-rich environments by learning from expert interventions during navigation. By integrating error backtrack functions and deep convolutional neural networks, the system maps raw images directly to control commands, achieving robust generalization across diverse real-world scenarios without explicit pedestrian modeling, with performance improving over iterative learning rounds.

ABSTRACT

Deep neural networks trained on demonstrations of human actions give robot the ability to perform self-driving on the road. However, navigation in a pedestrian-rich environment, such as a campus setup, is still challenging---one needs to take frequent interventions to the robot and take control over the robot from early steps leading to a mistake. An arduous burden is, hence, placed on the learning framework design and data acquisition. In this paper, we propose a new learning-from-intervention Dataset Aggregation (DAgger) algorithm to overcome the limitations brought by applying imitation learning to navigation in the pedestrian-rich environment. Our new learning algorithm implements an error backtrack function that is able to effectively learn from expert interventions. Combining our new learning algorithm with deep convolutional neural networks and a hierarchically-nested policy-selection mechanism, we show that our robot is able to map pixels direct to control commands and navigate successfully in real world without explicitly modeling the pedestrian behaviors or the world model.

Motivation & Objective

• To address the challenge of training robots to navigate safely in complex, pedestrian-rich environments like university campuses, where traditional imitation learning fails due to distributional shift.
• To reduce reliance on extensive pre-collected demonstration data by enabling real-time, interactive learning from expert interventions during deployment.
• To develop a scalable and efficient imitation learning framework that improves policy generalization through iterative data aggregation from human corrections.
• To enable end-to-end navigation from raw RGB images to control commands without explicit world or pedestrian behavior modeling.
• To demonstrate robust, real-time performance on a physical robot platform in diverse, previously unseen real-world environments.

Proposed method

• Proposes a novel Learn-from-Intervention DAgger algorithm that incorporates an error backtrack function to learn from expert interventions when the robot makes mistakes.
• Uses a hierarchical policy-selection mechanism to dynamically select control policies based on environmental context and observed states.
• Employs deep convolutional neural networks (e.g., ResNet) to map raw pixel inputs directly to steering and speed control commands.
• Implements a fixed-length data queue to store recent states and predicted actions, which is updated only when expert intervention occurs.
• Performs real-time inference on a Jetson TX2 at 10 FPS, converting network outputs into PWM signals for actuation.
• Aggregates new intervention data iteratively, with data collection diminishing as policy performance improves.

Experimental results

Research questions

• RQ1 Can a robot learn to navigate in pedestrian-rich environments by learning from expert interventions rather than relying solely on pre-collected demonstrations?
• RQ2 How does the Learn-from-Intervention DAgger algorithm improve policy generalization compared to standard behavior cloning or traditional DAgger?
• RQ3 To what extent can a vision-based, end-to-end policy map raw images to control commands without explicit modeling of pedestrian dynamics?
• RQ4 How does the frequency of expert interventions change over time, and what does this indicate about policy improvement?
• RQ5 Can the trained policy generalize to previously unseen environments and scenarios not included in the training data?

Key findings

• The number of successful navigation attempts increased with each iteration of the Learn-from-Intervention DAgger process, indicating improved policy generalization across all scenarios.
• Time Without Intervention (TWI) for path following and pedestrian following tasks increased significantly over iterations, demonstrating longer autonomous operation periods.
• The size of the training dataset converged after five iterations per scenario, indicating diminishing returns in data collection as policy performance improved.
• The system achieved real-time inference at 10 FPS on a Jetson TX2, enabling stable and responsive control in real-world conditions.
• The robot successfully navigated in four previously unseen campus regions, demonstrating strong generalization beyond the training data.
• The framework enabled effective learning from minimal, targeted expert corrections, reducing data collection burden while maintaining high performance.