[论文解读] Grand Challenges of Traceability: The Next Ten Years

Giuliano Antoniol, Jane Cleland‐Huang+2 more|arXiv (Cornell University)|Oct 9, 2017

Software Engineering Research参考文献 4被引用 12

一句话总结

本文识别并应对未来十年软件可追溯性领域面临的重大挑战，提出可操作的策略以提升数据集多样性、验证性和反馈机制。文章倡导采用标准化数据格式，激励共享数据集（包括负面结果），并推动社区主导的数据集整理工作，以增强可追溯性研究的可重现性与跨领域适用性。

ABSTRACT

In 2007, the software and systems traceability community met at the first Natural Bridge symposium on the Grand Challenges of Traceability to establish and address research goals for achieving effective, trustworthy, and ubiquitous traceability. Ten years later, in 2017, the community came together to evaluate a decade of progress towards achieving these goals. These proceedings document some of that progress. They include a series of short position papers, representing current work in the community organized across four process axes of traceability practice. The sessions covered topics from Trace Strategizing, Trace Link Creation and Evolution, Trace Link Usage, real-world applications of Traceability, and Traceability Datasets and benchmarks. Two breakout groups focused on the importance of creating and sharing traceability datasets within the research community, and discussed challenges related to the adoption of tracing techniques in industrial practice. Members of the research community are engaged in many active, ongoing, and impactful research projects. Our hope is that ten years from now we will be able to look back at a productive decade of research and claim that we have achieved the overarching Grand Challenge of Traceability, which seeks for traceability to be always present, built into the engineering process, and for it to have "effectively disappeared without a trace". We hope that others will see the potential that traceability has for empowering software and systems engineers to develop higher-quality products at increasing levels of complexity and scale, and that they will join the active community of Software and Systems traceability researchers as we move forward into the next decade of research.

研究动机与目标

• 解决可追溯性研究中缺乏多样化、可重用且标准化数据集的问题。
• 通过专家验证与复制研究提升数据质量与可信度。
• 通过激励数据共享、反馈与整理工作，加强社区协作。
• 通过投资标准化格式与元数据，提升跨领域互操作性。
• 通过基准测试与复制包支持可追溯性研究的长期可持续性。

提出的方法

• 提出由社区主导的数据集整理、匿名化与打包框架，以确保其可重用性。
• 倡导颁发成果物徽章与奖项，激励研究人员共享数据集，包括负面结果。
• 开展多机构参与的开源研究，以验证可追溯性链接并提升可靠性。
• 建立数据集的元数据标准，以记录评审流程、专家参与情况与数据来源。
• 鼓励发布复制包与标准化报告，以实现对新型可追溯性技术的公平比较。
• 呼吁针对数据验证的专项资助提案，以增加高质量真实数据集的数量。

实验结果

研究问题

• RQ1 可追溯性研究社区如何提升不同领域间基准数据集的多样性与可重用性？
• RQ2 哪些机制可确保可追溯性真实数据集的质量与专家验证？
• RQ3 如何激励研究人员不仅分享正面结果，也共享负面结果与复制包？
• RQ4 标准化格式与元数据在提升可追溯性研究的互操作性与可重现性方面发挥何种作用？
• RQ5 如何通过社区范围的数据集整理与反馈系统提升可追溯性数据集的长期可持续性？

主要发现

• 缺乏多样化且标准化的数据集限制了可追溯性技术在不同领域间的泛化能力。
• 许多现有数据集缺乏专家验证，降低了其可靠性与基准测试的实用性。
• 通过徽章、奖项与特刊征稿等方式激励数据共享，可显著提升数据集贡献率与透明度。
• 在多个机构开展的复制研究可提升可追溯性链接预测的可信度与可重现性。
• 在出版物中包含负面结果与复制包可增强方法论严谨性，并支持迭代改进。
• 标准化的元数据与打包指南对于有效整理、匿名化与重用数据集至关重要。