[Paper Review] Combinatorial Testing for Deep Learning Systems

Lei Ma, Fuyuan Zhang+5 more|arXiv (Cornell University)|Jun 20, 2018

Adversarial Robustness in Machine LearningComputer Science39 references59 citations

TL;DR

This paper explores applying combinatorial testing (CT) to deep learning (DL) systems, proposing DL-specific CT coverage criteria and a CT-guided test generation method to assess local robustness and adversarial vulnerability.

ABSTRACT

Deep learning (DL) has achieved remarkable progress over the past decade and been widely applied to many safety-critical applications. However, the robustness of DL systems recently receives great concerns, such as adversarial examples against computer vision systems, which could potentially result in severe consequences. Adopting testing techniques could help to evaluate the robustness of a DL system and therefore detect vulnerabilities at an early stage. The main challenge of testing such systems is that its runtime state space is too large: if we view each neuron as a runtime state for DL, then a DL system often contains massive states, rendering testing each state almost impossible. For traditional software, combinatorial testing (CT) is an effective testing technique to reduce the testing space while obtaining relatively high defect detection abilities. In this paper, we perform an exploratory study of CT on DL systems. We adapt the concept in CT and propose a set of coverage criteria for DL systems, as well as a CT coverage guided test generation technique. Our evaluation demonstrates that CT provides a promising avenue for testing DL systems. We further pose several open questions and interesting directions for combinatorial testing of DL systems.

Motivation & Objective

• Motivate testing DL systems due to robustness concerns (e.g., adversarial examples) in safety-critical applications.
• Adapt combinatorial testing to DL by defining neuron-activation based CT criteria.
• Propose a CT-guided test generation technique to systematically cover CT targets in DL layers.
• Demonstrate the usefulness of CT for robustness testing via empirical evaluation on MNIST models.

Proposed method

• Define neuron-activation configurations based on neuron outputs split by 0.
• Introduce t-way combination sparse and dense coverage for neuron sets within a layer.
• Extend CT to (p, t)-completeness coverage to quantify layer-wide CT coverage.
• Develop CT Coverage Guided TestGen algorithm that iteratively covers CT targets across DL layers using constrained test generation (LP-based in this study).
• Implement DeepCT framework using Keras/TensorFlow and linear programming (CPLEX) for test generation.

Experimental results

Research questions

• RQ1 Can CT concepts be adapted to DL to reduce testing space while preserving robustness detection capability?
• RQ2 Do DL-specific CT coverage criteria effectively guide test generation to reveal local robustness issues and adversarial examples?
• RQ3 How does CT-based testing compare with random testing in terms of coverage and adversarial detection on DL models?

Key findings

• CT coverage criteria yield high 2-way coverage as layers are analyzed, outperforming random testing.
• For DNNs on MNIST, CT-based testing achieves up to 97.81% 2-way sparse coverage and 99.21% 2-way dense coverage across deeper layers, with substantial reductions in test count (~4k–13k tests) compared to random testing.
• CT-based testing detects adversarial examples where random testing may miss, especially when covering early layers (L1–L3).
• Random testing shows limited 2-way coverage (e.g., 2.28% sparse for DNN1) and weaker completeness, while DeepCT achieves higher coverage with fewer tests.
• CT guidance indicates that different layers contribute differently to robustness detection, suggesting focal CT targeting per-layer.