[论文解读] Metamorphic Testing: A New Approach for Generating Next Test Cases

Tsong Yueh Chen, Shing-Chi Cheung+1 more|arXiv (Cornell University)|Feb 28, 2020

Software Testing and Debugging Techniques被引用 461

一句话总结

引入变换测试以从成功的测试用例派生新的测试用例，以揭示隐藏的软件错误，并在没有完整测试准则的情况下运行。

ABSTRACT

In software testing, a set of test cases is constructed according to some predefined selection criteria. The software is then examined against these test cases. Three interesting observations have been made on the current artifacts of software testing. Firstly, an error-revealing test case is considered useful while a successful test case which does not reveal software errors is usually not further investigated. Whether these successful test cases still contain useful information for revealing software errors has not been properly studied. Secondly, no matter how extensive the testing has been conducted in the development phase, errors may still exist in the software [5]. These errors, if left undetected, may eventually cause damage to the production system. The study of techniques for uncovering software errors in the production phase is seldom addressed in the literature. Thirdly, as indicated by Weyuker in [6], the availability of test oracles is pragmatically unattainable in most situations. However, the availability of test oracles is generally assumed in conventional software testing techniques. In this paper, we propose a novel test case selection technique that derives new test cases from the successful ones. The selection aims at revealing software errors that are possibly left undetected in successful test cases which may be generated using some existing strategies. As such, the proposed technique augments the effectiveness of existing test selection strategies. The technique also helps uncover software errors in the production phase and can be used in the absence of test oracles.

研究动机与目标

• 激发传统测试用例选择的局限性以及重复使用成功测试用例的价值。
• 提出将变换测试作为一种派生新测试用例的技术，旨在暴露尚未发现的错误。
• 展示变换测试如何增强现有测试用例选择策略并支持生产阶段的错误检测。
• 解决在许多实际场景中缺乏可靠的测试准则的问题。

提出的方法

• 将变换测试描述为从成功测试用例派生新测试用例。
• 解释变换关系如何引导后续测试用例的生成。
• 论证这种方法可以增强现有测试策略，并且在测试准则不可用时也能运行。
• 将变换测试定位为揭示可能被常规测试遗漏的错误的一种手段。

实验结果

研究问题

• RQ1 如何从成功的测试用例派生新的测试用例以揭示隐藏的错误？
• RQ2 变换测试能否增强传统的测试用例选择策略？
• RQ3 在缺少测试准则时，变换测试如何帮助生产阶段的错误检测？

主要发现

• 变换测试从成功测试派生额外的测试用例，以揭示现有策略未能揭示的错误。
• 该技术补充并增强现有测试用例选择方法的效果。
• 它为生产阶段的错误检测提供了实际的方法，在该阶段测试准则通常不可用。