[Paper Review] Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car

Mariusz Bojarski, Philip Yeres+5 more|arXiv (Cornell University)|Apr 25, 2017

Explainable Artificial Intelligence (XAI)Computer Science7 references242 citations

TL;DR

The paper presents PilotNet, an end-to-end CNN for steering, and introduces a saliency visualization method to identify image regions that most influence steering decisions, showing PilotNet learns meaningful road cues.

ABSTRACT

As part of a complete software stack for autonomous driving, NVIDIA has created a neural-network-based system, known as PilotNet, which outputs steering angles given images of the road ahead. PilotNet is trained using road images paired with the steering angles generated by a human driving a data-collection car. It derives the necessary domain knowledge by observing human drivers. This eliminates the need for human engineers to anticipate what is important in an image and foresee all the necessary rules for safe driving. Road tests demonstrated that PilotNet can successfully perform lane keeping in a wide variety of driving conditions, regardless of whether lane markings are present or not. The goal of the work described here is to explain what PilotNet learns and how it makes its decisions. To this end we developed a method for determining which elements in the road image most influence PilotNet's steering decision. Results show that PilotNet indeed learns to recognize relevant objects on the road. In addition to learning the obvious features such as lane markings, edges of roads, and other cars, PilotNet learns more subtle features that would be hard to anticipate and program by engineers, for example, bushes lining the edge of the road and atypical vehicle classes.

Motivation & Objective

• Demonstrate an end-to-end CNN that outputs steering commands from front-camera images trained via human-driven data.
• Develop a simple, fast saliency method to identify image regions driving PilotNet's steering decisions.
• Show that the learned features align with human driving cues and reveal non-obvious yet informative road elements.

Proposed method

• Describe PilotNet architecture (9 layers: normalization, 5 conv, 3 fully connected) and input preprocessing (YUV channels).
• Train end-to-end using image-steering pairs from a data-collection car with data augmentation simulating off-center/orientation shifts.
• Introduce a saliency visualization algorithm that propagates activations from high-level feature maps to input image to create a visualization mask.
• Overlay visualization masks on input images to identify salient objects influencing steering.
• Validate saliency regions by perturbation tests: translating salient vs non-salient regions to observe steering changes.

Experimental results

Research questions

• RQ1 What regions of the road image most influence PilotNet's steering output?
• RQ2 Do the learned salient regions correspond to meaningful driving cues (lanes, road edges, other vehicles)?
• RQ3 Can saliency visualization reveal subtle features learned by PilotNet that are not explicitly programmed?
• RQ4 Do perturbation experiments confirm that salient regions control steering as predicted by the visualization?
• RQ5 How does end-to-end learning affect where the network attends compared to hand-engineered rules?

Key findings

• PilotNet learns to recognize relevant road objects such as lane markings, road edges, and other cars.
• Saliency visualization reveals additional features like bushes at road edges and atypical vehicle classes that humans might not manually code.
• Shifting only salient regions produces a significant change in steering, similar to shifting the whole image, while shifting non-salient regions has little effect.
• PilotNet’s attention patterns align with human driving cues, supporting trust in its decisions.
• The visualization method provides near-pixel-level insight with fast execution suitable for in-car monitoring.