[论文解读] A survey on test practitioners' awareness of test smells

Nildo Silva, Larissa Rocha+2 more|arXiv (Cornell University)|Mar 12, 2020

Software Engineering Research参考文献 18被引用 14

一句话总结

本研究通过调查巴西公司中的60名测试从业者，探究了测试从业者在测试开发过程中无意识引入测试异味（test smells）的情况。研究发现，即使遵循标准化实践，经验丰富的开发人员仍频繁使用导致测试异味的反模式（anti-patterns），且专业经验并未显著降低异味引入的概率。

ABSTRACT

Developing test code may be a time-consuming task that usually requires much effort and cost, especially when it is done manually. Besides, during this process, developers and testers are likely to adopt bad design choices, which may lead to the introduction of the so-called test smells in test code. Test smells are bad solutions to either implement or design test code. As the test code with test smells increases in size, these tests might become more complex, and as a consequence, much harder to understand and evolve correctly. Therefore, test smells may have a negative impact on the quality and maintenance of test code and may also harm the whole software testing activities. In this context, this study aims to understand whether test professionals non-intentionally insert test smells. We carried out an expert survey to analyze the usage frequency of a set of test smells. Sixty professionals from different companies participated in the survey. We selected 14 widely studied smells from the literature, which are also implemented in existing test smell detection tools. The yielded results indicate that experienced professionals introduce test smells during their daily programming tasks, even when they are using standardized practices from their companies, and not only for their personal assumptions. Another relevant evidence was that developers' professional experience can not be considered as a root-cause for the insertion of test smells in test code.

研究动机与目标

• 了解测试从业者是否在测试开发过程中无意识地引入测试异味。
• 检验专业经验是否能降低引入测试异味的可能性。
• 识别在真实测试工作流中导致测试异味插入的常见开发实践。
• 评估行业专业人士中特定测试异味相关实践的频率与普遍性。
• 通过识别测试异味引入的根本原因（超越个人专业知识），为提升测试质量提供建议。

提出的方法

• 对来自巴西不同公司的60名测试从业者开展了专家调查。
• 从文献中选取了14种广泛研究的测试异味，这些异味常见于测试异味检测工具中。
• 在调查中将测试异味呈现为开发实践，以避免偏见并提高理解度。
• 通过四名测试人员的预调查对问题进行优化，以减少歧义。
• 对开放性回答采用同行评审的编码方法，以确保建构效度。
• 将调查问卷从葡萄牙语翻译为英语，并由所有作者审阅，以保持一致性。

实验结果

研究问题

• RQ1 RQ1：从业者是否使用可能导致测试异味插入的测试用例设计实践？
• RQ2 RQ2：专业经验是否会影响测试异味的插入？
• RQ3 RQ3：从业者日常活动中哪些实践会导致测试异味的插入？
• RQ4 RQ4：某些测试异味是否更常与标准化或习惯性开发实践相关联？

主要发现

• 经验丰富的从业者即使遵循公司标准化实践，也频繁引入测试异味。
• 专业经验并未显著降低测试异味引入的频率，这挑战了‘专业知识可消除不良实践’的假设。
• 所选的14种测试异味均有至少部分从业者报告使用，表明这些有问题的模式在业界普遍存在且被广泛采纳。
• 测试异味的主要成因被确定为习惯性编码实践和对组织标准的遵循，而非个人假设。
• 如‘编写不良的测试’、‘测试中的条件逻辑’和‘通用测试数据’等测试异味被频繁提及为常见实践。
• 研究未发现经验水平与减少异味插入之间存在显著相关性，表明仅靠经验不足以解决问题，可能需要额外的培训或工具支持。