[论文解读] Exploring Research Interest in Stack Overflow -- A Systematic Mapping Study and Quality Evaluation

Sarah Meldrum, Sherlock A. Licorish+1 more|arXiv (Cornell University)|Oct 23, 2020

Software Engineering Research参考文献 63被引用 6

一句话总结

本篇系统映射研究分析了265篇关于Stack Overflow的学术论文，以评估学术兴趣、研究主题、研究方法、研究贡献及研究质量。研究发现，学术界对Stack Overflow的关注度持续上升，尤其集中在社区动态和技术问题方面，但平均仅满足58%的质量标准，凸显未来研究中亟需加强验证工作。

ABSTRACT

Platforms such as Stack Overflow are available for software practitioners to solicit solutions to their challenges and knowledge needs. The practices therein have in recent times however triggered quality related concerns. This is a noteworthy issue when considering that the Stack Overflow platform is used by numerous software developers. Academic research tends to provide validation for the practices and processes employed by Stack Overflow and other such forums. However, previous work did not review the scale of scientific attention that is given to this cause. Continuing from our preliminary work, we conducted a Systematic Mapping study involving 265 papers from six relevant databases to address this gap. In this work, we explored the level of academic interest Stack Overflow has generated, the publication venues that are targeted, the topics that are studied, approaches used, types of contributions and the quality of the publications that are written about Stack Overflow. Outcomes show that Stack Overflow has attracted increasing research interest over the years, with topics relating to both community dynamics and human factors, and technical issues. In addition, research studies have been largely evaluative or proposed solutions; however, the latter approach tends to lack validation. The contributions of these studies are often techniques or answers to a specific problem. Evaluating the quality of all studies that were dedicated to software programming (58 papers), our outcomes show that on average only 58% of the developed quality criteria were met. Notwithstanding that research is continually aiming to understand Stack Overflow and other similar communities, further investigations are required to validate such studies and the solutions they propose.

研究动机与目标

• 调查近年来出版物中关于Stack Overflow作为研究主题的学术关注度水平。
• 识别发表在相关期刊和会议上的主要出版渠道、研究主题及采用的研究方法。
• 评估聚焦于软件编程社区（尤其是以Stack Overflow为中心）的研究质量。
• 评估文献中所提出解决方案的实证验证程度。
• 识别当前研究中的空白，并为未来以验证为重点的研究提出建议方向。

提出的方法

• 通过六个主要学术数据库开展系统映射研究，以识别与Stack Overflow相关的出版物。
• 应用预设的纳入与排除标准，筛选出265篇论文的相关性与质量。
• 根据研究主题、出版渠道、研究方法及研究贡献类型对研究进行分类。
• 使用一套标准化的15项质量标准，评估58篇聚焦于软件编程的研究的质量。
• 计算在评估的研究中，质量标准平均达成的百分比。
• 采用主题分析法，识别文献中反复出现的研究主题与方法论趋势。

实验结果

研究问题

• RQ1 Stack Overflow的学术关注度随时间如何变化？哪些出版渠道被最频繁使用？
• RQ2 与Stack Overflow相关的研究中，主导的研究主题与核心议题是什么？
• RQ3 在关于Stack Overflow的研究中，最常用的研究方法是什么？解决方案又是如何评估的？
• RQ4 文献中所提出解决方案的实证验证程度如何？
• RQ5 关于Stack Overflow的研究的质量指标与既定的研究质量标准相比如何？

主要发现

• 对Stack Overflow的学术关注度随时间显著增长，多个出版渠道的发表数量持续上升。
• 最常见的研究主题聚焦于社区动态、人为因素以及软件开发中的技术挑战。
• 大多数研究为评估性研究或提出解决方案，但在58篇评估论文中，平均仅满足58%的质量标准。
• 所提出的解决方案通常缺乏实证验证，表明在严谨评估实践方面存在明显缺口。
• 研究贡献通常为特定技术或针对特定问题的解答，普遍缺乏可推广性或长期验证。
• 尽管关注度持续上升，但现有文献仍亟需更多方法论严谨且具备实证验证的研究，以应对Stack Overflow及类似平台的研究需求。