[Paper Review] Driving Policy Transfer via Modularity and Abstraction

Matthias Müller, Alexey Dosovitskiy+2 more|arXiv (Cornell University)|Apr 25, 2018

Autonomous Vehicle Technology and Safety122 citations

TL;DR

The paper shows how a modular architecture (perception, policy, low-level control) can transfer driving policies learned in simulation directly to a real 1/5-scale truck without fine-tuning, by operating on semantic maps and waypoints rather than raw images or motor commands.

ABSTRACT

End-to-end approaches to autonomous driving have high sample complexity and are difficult to scale to realistic urban driving. Simulation can help end-to-end driving systems by providing a cheap, safe, and diverse training environment. Yet training driving policies in simulation brings up the problem of transferring such policies to the real world. We present an approach to transferring driving policies from simulation to reality via modularity and abstraction. Our approach is inspired by classic driving systems and aims to combine the benefits of modular architectures and end-to-end deep learning approaches. The key idea is to encapsulate the driving policy such that it is not directly exposed to raw perceptual input or low-level vehicle dynamics. We evaluate the presented approach in simulated urban environments and in the real world. In particular, we transfer a driving policy trained in simulation to a 1/5-scale robotic truck that is deployed in a variety of conditions, with no finetuning, on two continents. The supplementary video can be viewed at https://youtu.be/BrMDJqI6H5U

Motivation & Objective

• Motivate and address the reality gap in autonomous driving by leveraging modularity and abstraction.
• Propose a three-stage architecture that isolates perception, policy, and control to ease sim-to-real transfer.
• Train perception on real-world segmentation data and train driving policy entirely in simulation using realistic perception outputs.
• Demonstrate transfer from simulation to a physical vehicle across diverse environments and conditions.
• Investigate how semantic representations and waypoint outputs contribute to robust transfer across domains.

Proposed method

• Three-module architecture: perception (encoder-decoder producing per-pixel road/non-road segmentation), driving policy (maps segmentation to local waypoint plan), and low-level controller (PID-based to follow waypoints).
• Perception is trained on Cityscapes for binary road segmentation using ERFNet and cross-entropy loss.
• Driving policy is trained in CARLA with conditional imitation learning (CIL) to output two waypoints encoded by distance and relative angle, conditioned on high-level commands (left/straight/right).
• Policy is trained on segmentation outputs that include realistic noise (no ground-truth segmentation) to mimic real perception imperfections.
• Training uses simulation data (28 hours) with an expert planner and a PID follower; data augmentation and weather variability are applied.
• Control uses separate PID controllers for throttle and steering based on waypoint angles (φ1) and target speeds.

Experimental results

Research questions

• RQ1 Can a modular perception-policy-control architecture enable direct sim-to-real transfer of driving policies without fine-tuning?
• RQ2 Does abstracting perception to semantic segmentation and driving to waypoint outputs improve generalization across environments and weather conditions?
• RQ3 How does training with noisy segmentation outputs affect real-world transfer performance?
• RQ4 What is the comparative performance of modular, waypoint-based policies versus end-to-end, image-based policies under domain shift?

Key findings

• The modular approach outperforms monolithic end-to-end baselines in simulation under unseen towns and weather conditions.
• In simulation, waypoint-based predictions from segmentation generalize better to new towns and weather than image-to-control or image-to-waypoint baselines.
• In the real world, the modular policy achieves 82% success without data augmentation and 100% with augmentation across three routes on a 1/5-scale truck.
• End-to-end image-based policies trained on color images fail to generalize well to the real world even with augmentation or domain randomization.
• The real-robot experiments demonstrate transfer from simulation to reality without finetuning, with only Cityscapes data used for perception training.
• The physical vehicle completed all three long routes with only a few infractions, including one severe infraction requiring intervention.