[Paper Review] Deep Imitation Learning for Autonomous Driving in Generic Urban Scenarios with Enhanced Safety

Jianyu Chen, Bodi Yuan+1 more|arXiv (Cornell University)|Mar 2, 2019

Autonomous Vehicle Technology and Safety27 references32 citations

TL;DR

The paper presents a deep imitation learning framework for urban driving using a bird-view input to predict future trajectories, augmented with a safety controller based on safe set theory to ensure safe test-time operation in simulation.

ABSTRACT

The decision and planning system for autonomous driving in urban environments is hard to design. Most current methods manually design the driving policy, which can be expensive to develop and maintain at scale. Instead, with imitation learning we only need to collect data and the computer will learn and improve the driving policy automatically. However, existing imitation learning methods for autonomous driving are hardly performing well for complex urban scenarios. Moreover, the safety is not guaranteed when we use a deep neural network policy. In this paper, we proposed a framework to learn the driving policy in urban scenarios efficiently given offline connected driving data, with a safety controller incorporated to guarantee safety at test time. The experiments show that our method can achieve high performance in realistic simulations of urban driving scenarios.

Motivation & Objective

• Motivate learning decision-making and planning for urban autonomous driving without hand-crafted policies.
• Propose a bird-view observation representation to improve sample efficiency and generalization.
• Learn a deep policy to predict planned trajectories from offline expert data.
• Incorporate a safety controller to guarantee safety during test time without predicting future obstacle motions.

Proposed method

• Use a bird-view input that encodes HD map, routing, traffic lights, past objects, and past ego states.
• Output a planned trajectory over a horizon H instead of direct control commands.
• Train a CNN (VGG-like) to map bird-view inputs to trajectory points with a displacement-based loss.
• Augment data by injecting noise during data collection to reduce covariate shift.
• Integrate a safety controller based on safe set theory that projects the planner output into a safe control set via quadratic programming.

Experimental results

Research questions

• RQ1 Can imitation learning from offline urban driving data yield a high-performance policy for generic urban scenarios?
• RQ2 Does the bird-view representation reduce sample complexity and improve generalization across urban layouts?
• RQ3 Does incorporating a safety controller ensure test-time safety without requiring obstacle trajectory predictions?
• RQ4 How does data augmentation affect closed-loop performance and recovery from perturbations?

Key findings

• The final model with data augmentation and safety controller achieves near-perfect success in intersection and roundabout trials (Intersection: 100%; Roundabout: 96%).
• Open-loop displacement error improves with augmentation and safety components, compared to non-augmented baselines (e.g., 0.16 m vs 0.44 m in new town for the base model).
• The safety controller guarantees safer behavior, reducing infractions (out-of-lane and collision) compared to other learning-based methods in Town03 and Town01.
• Data augmentation significantly improves recovery from abnormal states and enhances policy robustness in complex urban scenarios.
• The approach yields competitive or superior performance to modular pipelines and other learning methods in closed-loop CARLA experiments.