[论文解读] Towards Developing and Analysing Metric-Based Software Defect Severity Prediction Model

Umamaheswara Sharma B, Ravichandra Sadam|arXiv (Cornell University)|Oct 10, 2022

Software Engineering Research被引用 5

一句话总结

本文提出一种基于决策树的自训练半监督学习模型，用于预测软件模块的缺陷严重性，利用有标签和无标签数据。该方法通过迭代地为无标签实例分配高置信度伪标签，提升预测性能，降低系统风险和剩余服务时间，新颖的项目特定指标展示了切实的项目级效益。

ABSTRACT

In a critical software system, the testers have to spend an enormous amount of time and effort to maintain the software due to the continuous occurrence of defects. Among such defects, some severe defects may adversely affect the software. To reduce the time and effort of a tester, many machine learning models have been proposed in the literature, which use the documented defect reports to automatically predict the severity of the defective software modules. In contrast to the traditional approaches, in this work we propose a metric-based software defect severity prediction (SDSP) model that uses a self-training semi-supervised learning approach to classify the severity of the defective software modules. The approach is constructed on a mixture of unlabelled and labelled defect severity data. The self-training works on the basis of a decision tree classifier to assign the pseudo-class labels to the unlabelled instances. The predictions are promising since the self-training successfully assigns the suitable class labels to the unlabelled instances. On the other hand, numerous research studies have covered proposing prediction approaches as well as the methodological aspects of defect severity prediction models, the gap in estimating project attributes from the prediction model remains unresolved. To bridge the gap, we propose five project specific measures such as the Risk-Factor (RF), the Percent of Saved Budget (PSB), the Loss in the Saved Budget (LSB), the Remaining Service Time (RST) and Gratuitous Service Time (GST) to capture project outcomes from the predictions. Similar to the traditional measures, these measures are also calculated from the observed confusion matrix. These measures are used to analyse the impact that the prediction model has on the software project.

研究动机与目标

• 为填补缺陷严重性预测模型在估算项目特定结果方面的空白。
• 开发一种基于自训练半监督学习的指标化软件缺陷严重性预测（SDSP）模型。
• 通过ADASYN过采样技术，提升在缺陷严重性数据不平衡情况下的预测性能。
• 引入五种新的项目特定评估指标（RF、PSB、LSB、RST、GST），以评估预测在现实世界中的影响。
• 在AEEEM代码库中的真实软件系统上对模型进行实证验证。

提出的方法

• 该模型采用基于决策树作为基础分类器的自训练半监督学习框架。
• 无标签实例根据模型置信度被分配伪标签，使用阈值（Γ = 0.99）筛选高置信度预测结果。
• 通过ADASYN过采样技术对有标签数据集进行平衡，以缓解缺陷严重性类别之间的类别不平衡问题。
• 迭代过程中将有标签和伪标签数据合并，用于在每次迭代中重新训练分类器。
• 性能通过传统指标（准确率、F-measure）和基于混淆矩阵推导出的新颖项目特定指标进行评估。
• 在AEEEM代码库中的四个开源软件系统上进行评估，使用527个无标签实例。

实验结果

研究问题

• RQ1 自训练半监督学习模型能否有效利用有标签和无标签数据混合，对软件缺陷严重性进行分类？
• RQ2 在缺陷严重性数据不平衡的情况下，所提出的模型与传统监督方法相比，其预测性能如何？
• RQ3 所提出的项目特定指标（RF、PSB、LSB、RST、GST）在多大程度上反映了缺陷严重性预测在现实世界中的实际影响？
• RQ4 接受阈值（Γ）的选择如何影响模型的性能和泛化能力？
• RQ5 所提出的模型能否在提升预算效率的同时，降低系统风险和维护服务时间？

主要发现

• 自训练模型成功地为无标签实例分配了合适的伪标签，提升了分类性能。
• 平均而言，该模型降低了系统故障风险，并减少了缺陷修复所需的剩余服务时间。
• 风险因子（RF）下降，表明由于误报率减少，项目风险降低。
• 节省预算的百分比（PSB）上升，而节省预算的损失（LSB）下降，反映出成本效率的提升。
• 剩余服务时间（RST）和无偿服务时间（GST）减少，表明缺陷修复更加迅速且高效。
• 该模型在所有四个评估的软件系统中均表现出一致的性能提升，尤其通过ADASYN过采样技术显著改善了对少数严重性类别（minority severity classes）的处理。