[Paper Review] Deep Neural Network Test Coverage: How Far Are We?

Junjie Chen, Ming Yan+3 more|arXiv (Cornell University)|Oct 10, 2020

Adversarial Robustness in Machine Learning51 references18 citations

TL;DR

This study systematically evaluates whether popular deep neural network (DNN) test coverage metrics correlate with test effectiveness, controlling for test set size. It finds that structural coverage metrics generally fail to predict effectiveness, while non-structural coverage shows promise in over half the evaluated models, suggesting behavior-based metrics are more reliable for measuring test quality in DNN testing.

ABSTRACT

DNN testing is one of the most effective methods to guarantee the quality of DNN. In DNN testing, many test coverage metrics have been proposed to measure test effectiveness, including structural coverage and non-structural coverage (which are classified according to whether considering which structural elements are covered during testing). Those test coverage metrics are proposed based on the assumption: they are correlated with test effectiveness (i.e., the generation of adversarial test inputs or the error-revealing capability of test inputs in DNN testing studies). However, it is still unknown whether the assumption is tenable. In this work, we conducted the first extensive study to systematically validate the assumption by controlling for the size of test sets. In the study, we studied seven typical test coverage metrics based on 9 pairs of datasets and models with great diversity (including four pairs that have never been used to evaluate these test coverage metrics before). The results demonstrate that the assumption fails for structural coverage in general but holds for non-structural coverage on more than half of subjects, indicating that measuring the difference of DNN behaviors between test inputs and training data is more promising than measuring which structural elements are covered by test inputs for measuring test effectiveness. Even so, the current non-structural coverage metrics still can be improved from several aspects such as unfriendly parameters and unstable performance. That indicates that although a lot of test coverage metrics have been proposed before, there is still a lot of room for improvement of measuring test effectiveness in DNN testing, and our study has pointed out some promising directions.

Motivation & Objective

• To validate the widely held assumption that test coverage metrics correlate with test effectiveness in DNN testing.
• To examine whether structural and non-structural coverage metrics reliably predict the ability to generate adversarial inputs or reveal errors.
• To assess the impact of test set size on coverage-test effectiveness correlation, ensuring fair evaluation.
• To identify promising directions for improving test coverage metrics in DNN testing.
• To evaluate seven coverage metrics across nine diverse model-dataset pairs, including four previously untested combinations.

Proposed method

• The study conducted a controlled experiment with fixed test set sizes to isolate the effect of coverage on test effectiveness.
• Seven representative test coverage metrics were evaluated: four structural and three non-structural, based on different behavioral and structural criteria.
• Nine diverse DNN models and datasets were used, including four novel combinations not previously used for coverage metric evaluation.
• Test effectiveness was measured by the capability of test inputs to generate adversarial examples or reveal errors in the model.
• Statistical analysis was used to assess the correlation between coverage metrics and test effectiveness across all subjects.
• The study compared structural coverage (e.g., neuron, layer, or activation-based) with non-structural coverage (e.g., input space or behavior-based).

Experimental results

Research questions

• RQ1 Does structural test coverage correlate with test effectiveness in DNNs when test set size is controlled?
• RQ2 Does non-structural test coverage correlate with test effectiveness in DNNs under the same conditions?
• RQ3 How do different coverage metrics perform across diverse DNN models and datasets?
• RQ4 Are there specific limitations in current non-structural coverage metrics that hinder their reliability?
• RQ5 What are the most promising directions for improving test coverage metrics in DNN testing?

Key findings

• Structural coverage metrics showed no consistent correlation with test effectiveness across the evaluated models, indicating the assumption underlying their use is generally invalid.
• Non-structural coverage metrics demonstrated a significant correlation with test effectiveness in more than half of the evaluated subjects, suggesting greater promise for measuring test quality.
• The study found that measuring behavioral differences between test inputs and training data is more effective than tracking which structural components are activated.
• Current non-structural coverage metrics still suffer from unstable performance and parameter sensitivity, indicating room for improvement.
• The results suggest that future work should prioritize behavior-based coverage metrics over structural ones for DNN test effectiveness assessment.
• The evaluation of four previously untested model-dataset pairs revealed that coverage effectiveness varies significantly across diverse DNN architectures and data distributions.