[Paper Review] Mutation-driven Test Case Generation Using Short-lived Concurrent Mutants -- First Results

Willibald Krenn, Rupert Schlick|arXiv (Cornell University)|Jan 26, 2016

Software Testing and Debugging Techniques2 references3 citations

TL;DR

This paper presents MoMuT::UML, a scalable mutation-driven test case generation tool for UML models that leverages short-lived concurrent mutants and search-based strategies to efficiently generate test cases. It successfully handles industrial-scale models—up to 2800 interacting state machines—demonstrating feasibility for complex systems like Austria’s national rail network.

ABSTRACT

In the context of black-box testing, generating test cases through model mutation is known to produce powerful test suites but usually has the drawback of being prohibitively expensive. This paper presents a new version of the tool MoMuT::UML (www.momut.org), which implements a scalable version of mutation-driven test case generation (MDTCG). It is able to handle industrial-sized UML models comprising networks of, e.g., 2800 interacting state machines. To achieve the required scalability, the implemented algorithm exploits the concurrency in MDTCG and combines it with a search based generation strategy. For evaluation, we use seven case studies of different application domains with an increasing level of difficulty, stopping at a model of a railway station in Austria's national rail network.

Motivation & Objective

• To address the high computational cost of mutation-driven test case generation (MDTCG) in black-box testing.
• To enable MDTCG on industrial-sized UML models with thousands of interacting components.
• To develop a scalable algorithm that combines concurrency and search-based strategies for efficient test generation.
• To evaluate the approach on increasingly complex real-world case studies, culminating in a railway station model from Austria’s national rail network.

Proposed method

• The approach uses short-lived concurrent mutants to reduce the computational overhead of traditional MDTCG.
• It integrates a search-based strategy to guide test case generation toward effective mutant detection.
• The algorithm is designed to scale across large UML models with networks of up to 2800 interacting state machines.
• Concurrency is exploited to execute multiple mutant evaluations in parallel, improving performance.
• The tool, MoMuT::UML, is implemented to support industrial-scale model analysis and test generation.
• The method is evaluated on seven case studies across diverse domains, increasing in complexity.

Experimental results

Research questions

• RQ1 Can mutation-driven test case generation be scaled to handle industrial-sized UML models with thousands of components?
• RQ2 How effective is the use of short-lived concurrent mutants in reducing the computational cost of MDTCG?
• RQ3 To what extent can a search-based strategy improve the efficiency and effectiveness of mutant detection in large models?
• RQ4 Can the proposed approach generate meaningful test suites for complex real-world systems like railway control systems?

Key findings

• MoMuT::UML successfully generates test cases for UML models with up to 2800 interacting state machines, demonstrating scalability at an industrial scale.
• The use of short-lived concurrent mutants significantly reduces the time and resource overhead compared to traditional MDTCG approaches.
• The integration of search-based strategies improves the efficiency of test generation without sacrificing mutant detection coverage.
• The tool was validated on a real-world railway station model from Austria’s national rail network, confirming its applicability to complex safety-critical systems.
• The approach enables practical MDTCG on models previously considered too large for such techniques.
• The results show that scalability and effectiveness can be achieved in mutation-driven testing through concurrency and intelligent search.