[论文解读] Computer Games Are Serious Business and so is their Quality: Particularities of Software Testing in Game Development from the Perspective of Practitioners

Ronnie E. S. Santos, Cleyton Magalhães+4 more|arXiv (Cornell University)|Dec 12, 2018

Educational Games and Gamification参考文献 23被引用 3

一句话总结

本研究通过一项混合方法案例研究及对38名从业者的调查，探讨了计算机游戏开发中软件测试所面临的独特挑战。研究发现，游戏测试必须超越功能正确性，涵盖游戏平衡性、物理行为及娱乐质量的评估，凸显了针对游戏特定非功能性需求而定制的专门测试方法的必要性。

ABSTRACT

Over the last several decades, computer games started to have a significant impact on society. However, although a computer game is a type of software, the process to conceptualize, produce and deliver a game could involve unusual features. In software testing, for instance, studies demonstrated the hesitance of professionals to use automated testing techniques with games, due to the constant changes in requirements and design, and pointed out the need for creating testing tools that take into account the flexibility required for the game development process. Goal. This study aims to improve the current body of knowledge regarding software testing in game development and point out the existing particularities observed in software testing considering the development of a computer game. Method. A mixed-method approach based on a case study and an opinion survey was applied to collect quantitative and qualitative data from software professionals regarding the particularities of software testing in game development. Results. We analyzed over 70 messages posted on three well-established network of question-and-answer communities related to software engineering, software testing and game development and received answers of 38 professionals discussing differences between testing a computer game and a general software, and identified important aspects to be observed by practitioners in the process of planning, performing and reporting tests in this context. Conclusion. Considering computer games, software testing must focus not only on the common aspects of a general software, but also, track and investigate issues that could be related to game balance, game physics and entertainment related-aspects to guarantee the quality of computer games and a successful testing process.

研究动机与目标

• 识别并分析行业从业者所感知的计算机游戏开发中软件测试的特定挑战。
• 理解为何传统软件测试实践在游戏开发环境中往往失效。
• 探讨显著影响游戏测试的游戏性平衡与真实感等独特非功能性质量属性。
• 通过将游戏特定质量维度整合进测试生命周期，提出改进测试实践的框架。
• 通过从业者驱动的洞察，弥合通用软件测试方法论与游戏开发独特需求之间的差距。

提出的方法

• 结合对三个问答社区（Stack Overflow、Software Engineering Stack Exchange 和 GameDev Stack Exchange）中70余条消息的定性分析，开展混合方法研究。
• 向38名具有游戏开发与测试经验的软件专业人员发放调查问卷，收集其关于测试实践与挑战的见解。
• 采用主题分析法识别游戏测试中的重复模式与独特关切，重点关注其与通用软件测试的差异。
• 收集并分析从业者对自动化局限性、测试设计及游戏项目中报告机制的看法。
• 将研究发现整合进一个概念模型，突出游戏平衡性、物理特性与玩家体验等非功能性测试维度。
• 通过与现有软件测试及游戏开发文献交叉比对，验证研究发现。

实验结果

研究问题

• RQ1 从从业者视角来看，测试计算机游戏与通用软件的主要差异是什么？
• RQ2 为何软件测试专业人员在游戏开发中犹豫采用自动化测试技术？
• RQ3 除功能性外，哪些非功能性质量属性在评估计算机游戏质量时最为关键？
• RQ4 游戏特定因素（如平衡性、物理特性与娱乐价值）如何影响测试规划与执行？
• RQ5 测试人员在报告和追踪与游戏玩法及用户体验相关的问题时面临哪些独特挑战？

主要发现

• 从业者一致强调，游戏测试必须评估游戏平衡性、物理行为与玩家参与度等非功能性方面，这些在通用软件测试中通常未被优先考虑。
• 由于游戏设计高度易变且游戏机制与内容频繁变更，自动化测试在游戏开发中常被回避。
• 游戏中的大部分测试工作集中于探索性测试与手动测试，而非脚本化或自动化测试用例，尤其是在早期开发阶段。
• 游戏开发中的测试报告通常包含对趣味性、挑战性与沉浸感的主观评估，这些难以量化但对质量至关重要。
• 从业者频繁将可利用性、进度节奏与人工智能行为等游戏特异性问题列为高优先级测试关注点。
• 现有测试工具与方法论在有效应对游戏开发的动态性、创造性和体验性方面存在显著缺口。