[Paper Review] Factors that Contribute to the Success of a Software Organisation's DevOps Environment: A Systematic Review

Ashley Gwangwadza, Ridewaan Hanslo|arXiv (Cornell University)|Nov 8, 2022

Software Engineering Techniques and Practices29 references4 citations

TL;DR

This systematic literature review identifies 15 success factors across four categories—Collaborative Culture, Organizational Aspects, Tooling and Technology, and Continuous Practices—that contribute to DevOps success in software organizations. The study proposes a comprehensive DevOps environment success factors model based on 33 reviewed articles (2015–2021), offering a structured framework for both research and practice in DevOps adoption and optimization.

ABSTRACT

This research assesses the aspects of software organizations' DevOps environments and identifies the factors contributing to these environments' success. DevOps is a recent concept, and many organizations are moving from old-style software development methods to agile approaches such as DevOps. However, there is no comprehensive information on what factors impact the success of the DevOps environment once organizations adopt it. This research focused on addressing this gap through a systematic literature review. The systematic review consisted of 33 articles from five selected search systems and databases from 2015 to 2021. Based on the included articles, 15 factors were identified and grouped into four categories: Collaborative Culture, Organizational Aspects, Tooling and Technology, and Continuous Practices. In addition, this research proposes a DevOps environment success factors model to potentially contribute to DevOps research and practice. Recommendations are made for additional research on the effectiveness of the proposed model and its success factors.

Motivation & Objective

• To identify and categorize factors contributing to the success of DevOps environments in software organizations.
• To address the lack of comprehensive, evidence-based insights on DevOps success factors post-adoption.
• To synthesize findings from existing literature to develop a structured model for DevOps success.
• To guide practitioners and researchers in understanding the multidimensional nature of DevOps implementation success.

Proposed method

• Conducted a systematic literature review using five academic databases and search systems.
• Applied predefined inclusion and exclusion criteria to screen 33 relevant articles published between 2015 and 2021.
• Performed thematic analysis to extract and categorize success factors into four overarching themes.
• Used qualitative synthesis to develop a conceptual model of DevOps environment success factors.
• Validated the model’s coherence and completeness through iterative review of the literature and factor clustering.
• Proposed a novel DevOps environment success factors model integrating identified factors into a unified framework.

Experimental results

Research questions

• RQ1 What factors contribute to the success of a software organization’s DevOps environment?
• RQ2 How are these success factors categorized across different dimensions of DevOps implementation?
• RQ3 What is the relative importance and interplay of cultural, organizational, technological, and practice-based factors in DevOps success?
• RQ4 How can a comprehensive model of DevOps success factors be synthesized from existing empirical evidence?
• RQ5 What are the implications of the proposed model for future research and organizational DevOps adoption?

Key findings

• Fifteen distinct success factors were identified and grouped into four categories: Collaborative Culture, Organizational Aspects, Tooling and Technology, and Continuous Practices.
• Collaborative Culture emerged as a foundational factor, emphasizing communication, trust, and shared responsibility across development and operations teams.
• Organizational Aspects such as leadership support, clear goals, and effective change management were consistently linked to DevOps success.
• Tooling and Technology factors, including automation tools and CI/CD pipelines, were essential but insufficient alone without cultural and process alignment.
• Continuous Practices such as monitoring, feedback loops, and iterative improvement were critical for sustaining DevOps performance over time.
• The proposed DevOps environment success factors model integrates these elements into a holistic framework for guiding organizational DevOps transformation.