[论文解读] Guidelines for including grey literature and conducting multivocal literature reviews in software engineering

Vahid Garousi, Michael Felderer+1 more|arXiv (Cornell University)|Jul 9, 2017

Software Engineering Techniques and Practices被引用 18

一句话总结

本文提出了在软件工程领域开展多源文献综述（MLRs）的全面、基于证据的指南，整合了正式出版物与灰色文献（如白皮书、博客文章）。基于系统性文献综述（SLR）的最佳实践、跨学科MLR经验以及真实的MLR案例研究，作者构建了一个涵盖规划、执行与报告阶段的结构化框架，使研究人员能够系统性地综合学术与实践知识。

ABSTRACT

Context: A Multivocal Literature Review (MLR) is a form of a Systematic Literature Review (SLR) which includes the grey literature (e.g., blog posts and white papers) in addition to the published (formal) literature (e.g., journal and conference papers). MLRs are useful for both researchers and practitioners since they provide summaries both the state-of-the art and -practice in a given area. Objective: There are several guidelines to conduct SLR studies in SE. However, given the facts that several phases of MLRs differ from those of traditional SLRs, for instance with respect to the search process and source quality assessment. Therefore, SLR guidelines are only partially useful for conducting MLR studies. Our goal in this paper is to present guidelines on how to conduct MLR studies in SE. Method: To develop the MLR guidelines, we benefit from three inputs: (1) existing SLR guidelines in SE, (2), a literature survey of MLR guidelines and experience papers in other fields, and (3) our own experiences in conducting several MLRs in SE. All derived guidelines are discussed in the context of three examples MLRs as running examples (two from SE and one MLR from the medical sciences). Results: The resulting guidelines cover all phases of conducting and reporting MLRs in SE from the planning phase, over conducting the review to the final reporting of the review. In particular, we believe that incorporating and adopting a vast set of recommendations from MLR guidelines and experience papers in other fields have enabled us to propose a set of guidelines with solid foundations. Conclusion: Having been developed on the basis of three types of solid experience and evidence, the provided MLR guidelines support researchers to effectively and efficiently conduct new MLRs in any area of SE.

研究动机与目标

• 解决在软件工程文献综述中系统性纳入灰色文献的指导空白。
• 开发一个稳健、基于证据的框架，用于开展超越传统系统性文献综述（SLRs）的多源文献综述（MLRs）。
• 支持研究人员与实践者在SE研究中综合学术与实践知识来源。
• 通过结构化指南确保MLR过程的方法论严谨性与透明度。

提出的方法

• 作者综合了软件工程领域现有系统性文献综述（SLR）实践的指南。
• 他们整合了其他学科（如管理学与医疗保健）中关于MLR的指南与经验论文的见解。
• 他们结合了自身在软件工程领域开展多项MLRs的实证经验。
• 通过三个案例研究对指南进行了验证与说明：其中两个案例来自软件工程，一个来自医学科学。
• 该框架覆盖了MLR的全部阶段：规划、检索策略、文献源选择、质量评估、数据提取、综合与报告。
• 该方法强调在纳入博客、白皮书等非传统来源时，保持透明度、可复现性与方法论严谨性。

实验结果

研究问题

• RQ1 如何在软件工程文献综述中系统性地识别、选择与评估灰色文献？
• RQ2 在软件工程中开展传统SLR与MLR的关键差异是什么？
• RQ3 在文献综述中纳入非正式或非同行评审来源时，如何保持方法论严谨性？
• RQ4 在软件工程中，MLR的规划、执行与报告的最佳实践是什么？
• RQ5 如何通过整合学术与实践知识来源，提升SE中文献综述的相关性与影响力？

主要发现

• 所提出的指南为在软件工程中开展MLRs提供了全面、分阶段的框架，涵盖规划、执行与报告。
• 纳入灰色文献显著丰富了综述内容，能够捕捉正式出版物中未体现的当前产业实践与新兴趋势。
• 指南基于三大坚实来源：既定的SLR实践、跨学科MLR经验以及真实的MLR案例研究。
• 通过为灰色文献来源量身定制的结构化质量评估标准，提升了MLRs的方法论严谨性。
• 该框架支持可复现性与透明度，使研究人员能够系统性地记录并证明其对非传统来源的纳入与评估。
• 该指南具有可迁移性，适用于软件工程领域内多样化的研究方向。