[Paper Review] DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars

Yuchi Tian, Kexin Pei+2 more|arXiv (Cornell University)|Aug 28, 2017

Adversarial Robustness in Machine Learning53 references220 citations

TL;DR

DeepTest automatically synthesizes realistic test inputs by applying image transformations to maximize neuron coverage and uses metamorphic-relations to detect erroneous DNN behaviors in autonomous driving models.

ABSTRACT

Recent advances in Deep Neural Networks (DNNs) have led to the development of DNN-driven autonomous cars that, using sensors like camera, LiDAR, etc., can drive without any human intervention. Most major manufacturers including Tesla, GM, Ford, BMW, and Waymo/Google are working on building and testing different types of autonomous vehicles. The lawmakers of several US states including California, Texas, and New York have passed new legislation to fast-track the process of testing and deployment of autonomous vehicles on their roads. However, despite their spectacular progress, DNNs, just like traditional software, often demonstrate incorrect or unexpected corner case behaviors that can lead to potentially fatal collisions. Several such real-world accidents involving autonomous cars have already happened including one which resulted in a fatality. Most existing testing techniques for DNN-driven vehicles are heavily dependent on the manual collection of test data under different driving conditions which become prohibitively expensive as the number of test conditions increases. In this paper, we design, implement and evaluate DeepTest, a systematic testing tool for automatically detecting erroneous behaviors of DNN-driven vehicles that can potentially lead to fatal crashes. First, our tool is designed to automatically generated test cases leveraging real-world changes in driving conditions like rain, fog, lighting conditions, etc. DeepTest systematically explores different parts of the DNN logic by generating test inputs that maximize the numbers of activated neurons. DeepTest found thousands of erroneous behaviors under different realistic driving conditions (e.g., blurring, rain, fog, etc.) many of which lead to potentially fatal crashes in three top performing DNNs in the Udacity self-driving car challenge.

Motivation & Objective

• Motivate safety-critical testing for DNN-driven autonomous cars and highlight limitations of manual data collection.
• Introduce neuron coverage as a guidance signal to explore DNN input-output space.
• Develop methods to synthesize realistic, transformation-based test inputs that expand neuron coverage.
• Propose metamorphic-relations as an automatic test oracle to detect erroneous corner-case behaviors.
• Evaluate the approach on Udacity self-driving models and release detected cases publicly.

Proposed method

• Define neuron coverage as the ratio of activated neurons to total neurons and use it to partition the DNN input space.
• Generate realistic synthetic test inputs by applying image transformations (brightness/contrast, blur, rain, fog, translation, rotation, scale, shear) to seed images.
• Propose a neuron-coverage guided greedy search to combine multiple transformations and maximize coverage.
• Use metamorphic relations between outputs on transformed inputs to automatically detect erroneous behaviors without detailed manual specifications.
• Implement DeepTest on three Udacity top-performing models (Chauffeur, Rambo, Epoch) using Keras/TensorFlow backbones and evaluate results.

Experimental results

Research questions

• RQ1 Does neuron coverage correlate with autonomous car outputs (steering angle and direction)?
• RQ2 Do different realistic image transformations activate different neurons in the DNNs?
• RQ3 Can combining transformations further increase neuron coverage and reveal more corner cases?
• RQ4 Can metamorphic relations serve as effective test oracles for detecting erroneous behaviors under transformed inputs?

Key findings

• Neuron coverage changes with different input-output pairs, showing statistically significant correlations with steering angle and direction across models.
• Different image transformations activate different neurons, with transformations generally increasing neuron coverage across models.
• Combining transformations further increases coverage, supporting a neuron-coverage guided search for test input synthesis.
• DeepTest discovered thousands of erroneous behaviors in three top Udacity DNN models, including potentially fatal scenarios, under realistic conditions like rain, fog, and blur.
• Synthetic test images can be used for retraining to improve DNN robustness, and the authors provide public access to detected erroneous behaviors and plan to release test images and DeepTest source.