[Paper Review] Testing Deep Neural Networks

Youcheng Sun, Xiaowei Huang+4 more|arXiv (Cornell University)|Mar 10, 2018

Adversarial Robustness in Machine LearningComputer Science81 references175 citations

TL;DR

This paper introduces four MC/DC-inspired white-box test criteria for DNNs, and demonstrates test case generation via concolic LP and gradient-based search to uncover adversarial behaviors while balancing coverage and cost.

ABSTRACT

Deep neural networks (DNNs) have a wide range of applications, and software employing them must be thoroughly tested, especially in safety-critical domains. However, traditional software test coverage metrics cannot be applied directly to DNNs. In this paper, inspired by the MC/DC coverage criterion, we propose a family of four novel test criteria that are tailored to structural features of DNNs and their semantics. We validate the criteria by demonstrating that the generated test inputs guided via our proposed coverage criteria are able to capture undesired behaviours in a DNN. Test cases are generated using a symbolic approach and a gradient-based heuristic search. By comparing them with existing methods, we show that our criteria achieve a balance between their ability to find bugs (proxied using adversarial examples) and the computational cost of test case generation. Our experiments are conducted on state-of-the-art DNNs obtained using popular open source datasets, including MNIST, CIFAR-10 and ImageNet.

Motivation & Objective

• Motivate the need for structured testing of DNNs in safety-critical domains.
• Develop four MC/DC-inspired coverage criteria tailored to DNN structure and semantics.
• Provide a white-box framework for test case generation guided by the proposed coverage criteria.
• Evaluate the criteria on MNIST, CIFAR-10, and ImageNet to study bug finding and testing efficiency.
• Enable analysis of DNN internal structure and robustness through generated adversarial examples.

Proposed method

• Define a DNN formalism with layers, activations, and feature sets.
• Introduce four coverage criteria: SS (Sign-Sign), VS (Value-Sign), SV (Sign-Value), VV (Value-Value).
• Model features as subsets of neurons across adjacent layers and define conditions/decisions via feature pairs.
• Use value functions g to quantify significant changes in feature values.
• Apply concolic testing (LP-based) to generate test cases that satisfy conditioning constraints.
• Develop a gradient-descent based test case generator to scale to large DNNs.

Experimental results

Research questions

• RQ1 Can MC/DC-inspired coverage criteria capture causal relationships in DNNs and guide effective test case generation?
• RQ2 How do SS, VS, SV, and VV criteria compare in uncovering adversarial examples and balancing computational cost?
• RQ3 Do the proposed criteria scale to large networks and datasets (MNIST, CIFAR-10, ImageNet)?
• RQ4 How do the new criteria relate to and improve upon existing neuron-based and safety coverage criteria?

Key findings

• The four criteria extend MC/DC concepts to DNNs and capture causal interactions across layers.
• SS coverage tests independent sign changes of condition features affecting decision features.
• VS, SV, and VV cover different combinations of value and sign changes to capture non-linear DNN behavior.
• Concolic LP-based test generation yields efficient test cases for small to medium DNNs, while the gradient-based method scales to large DNNs.
• Experiments on MNIST, CIFAR-10, and ImageNet show the criteria can guide bug finding (adversarial examples) and provide insights into DNN safety and internal structure.
• The proposed criteria impose stronger coverage than prior neuron-based criteria, offering richer testing at higher computational cost.