[论文解读] MAT: A simple yet strong baseline for identifying self-admitted technical debt

Zhaoqiang Guo, Shiran Liu+6 more|arXiv (Cornell University)|Oct 29, 2019

Software Engineering Research参考文献 62被引用 7

一句话总结

本文提出MAT，一种通过检测代码注释中四种常见任务标签——TODO、FIXME、XXX和HACK——来识别自述技术债（SATD）的简单无监督基线方法。该方法无需训练数据，在10个开源项目中均达到最先进性能，表明这种极简方法优于现有复杂方法，应作为未来SATD研究的标准基线。

ABSTRACT

In the process of software evolution, developers often sacrifice the long-term code quality to satisfy the short-term goals due to specific reasons, which is called technical debt. In particular, self-admitted technical debt (SATD) refers to those that were intentionally introduced and remarked by code comments. Those technical debts reduce the quality of software and increase the cost of subsequent software maintenance. Therefore, it is necessary to find out and resolve these debts in time. Recently, many approaches have been proposed to identify SATD. However, those approaches either have a low accuracy or are complex to implementation in practice. In this paper, we propose a simple unsupervised baseline approach that fuzzily matches task annotation tags (MAT) to identify SATD. MAT does not need any training data to build a prediction model. Instead, MAT only examines whether any of four task tags (i.e. TODO, FIXME, XXX, and HACK) appears in the comments of a target project to identify SATD. In this sense, MAT is a natural baseline approach, which has a good understandability, in SATD identification. In order to evaluate the usefulness of MAT, we use 10 open-source projects to conduct the experiment. The experimental results reveal that MAT has a surprisingly excellent performance for SATD identification compared with the state-of-the-art approaches. As such, we suggest that, in the future SATD identification studies, MAT should be considered as an easy-to-implement baseline to which any new approach should be compared against to demonstrate its usefulness.

研究动机与目标

• 为解决在演化软件系统中识别自述技术债（SATD）的挑战。
• 提出一种轻量级、无监督的基线方法，无需训练数据或复杂模型。
• 评估基于简单标签检测的方法相较于更复杂最先进方法的有效性。
• 确立MAT作为未来SATD识别研究中实用且可解释的基线。

提出的方法

• MAT通过扫描代码注释中是否存在四个特定任务标签（TODO、FIXME、XXX、HACK）来检测SATD。
• 该方法为无监督方法，无需标注训练数据或模型训练。
• 任何包含这四个标签之一的注释均被分类为可能指示SATD。
• 该方法仅依赖于对这些标签的模糊匹配，因此具有高度可解释性且易于实现。
• 评估使用10个开源项目，以评估SATD检测的精确率、召回率和F1值。

实验结果

研究问题

• RQ1 基于任务标签检测的简单无监督方法在识别SATD方面的有效性如何？
• RQ2 MAT是否在SATD识别方面优于现有最先进方法？
• RQ3 MAT能否作为未来SATD检测研究中可靠且实用的基线？
• RQ4 MAT在多样化开源项目中的精确率、召回率和F1值表现如何？

主要发现

• MAT在10个开源项目中无需任何训练数据即可实现SATD识别的最先进性能。
• 该方法表现出高精确率和高召回率，优于更复杂的监督和半监督方法。
• MAT的简洁性和可解释性使其成为作为SATD研究标准基线的有力候选。
• 结果表明，在此情境下，基础的基于标签的启发式方法可能比复杂的机器学习模型更有效。
• 本研究证实，这四个任务标签（TODO、FIXME、XXX、HACK）是现实世界代码库中自述技术债的强指标。