[Paper Review] Revealing the State of the Art of Large-Scale Agile Development Research: A Systematic Mapping Study

Oemer Uludag, Pascal Philipp+4 more|arXiv (Cornell University)|Jul 10, 2020

Software Engineering Techniques and Practices4 citations

TL;DR

This systematic mapping study analyzes 136 primary studies to map the state of the art in large-scale agile development research from 2007 to 2022. It identifies key research themes, classifies industry practices, and proposes a research agenda, offering a comprehensive overview of the field’s evolution and gaps for researchers and practitioners.

ABSTRACT

Context: Success with agile methods in the small scale has led to an increasing adoption also in large development undertakings and organizations. Recent years have also seen an increasing amount of primary research on the topic, as well as a number of systematic literature reviews. However, there is no systematic overview of the whole research field. Objective: This work identifies, classifies, and evaluates the state of the art of research in large-scale agile development. Method: We conducted a systematic mapping study and rigorously selected 136 studies. We designed a classification framework and extracted key information from the studies. We synthesized the obtained data and created an overview of the state of the art. Results: This work contributes with (i) a description of large-scale agile endeavors reported in the industry, (ii) a systematic map of existing research in the field, (iii) an overview of influential studies, (iv) an overview of the central research themes, and (v) a research agenda for future research. Conclusion: This study portrays the state of the art in large-scale agile development and offers researchers and practitioners a reflection of the past thirteen years of research and practice on the large-scale application of agile methods.

Motivation & Objective

• To provide a systematic overview of the current state of research in large-scale agile development, which has grown significantly but lacks a consolidated synthesis.
• To classify and analyze 136 primary studies to understand the scope, focus, and methodological trends in the field.
• To identify and describe the types of large-scale agile endeavors reported in industry practice.
• To highlight influential studies and recurring research themes in the literature.
• To propose a research agenda for future work based on identified gaps and emerging trends.

Proposed method

• Conducted a systematic mapping study using predefined inclusion and exclusion criteria to select 136 relevant primary studies from a broad literature base.
• Developed a comprehensive classification framework to categorize studies by research focus, context, methodology, and key concepts.
• Extracted and synthesized data from each study, including research questions, settings, methods, and findings.
• Mapped the distribution of studies across research themes and identified thematic clusters in large-scale agile development research.
• Evaluated the quality and methodological rigor of selected studies using established criteria for systematic literature reviews.
• Synthesized findings into a visual and analytical overview of the research landscape, including trends and gaps.

Experimental results

Research questions

• RQ1 What are the main research themes and topics explored in large-scale agile development research between 2007 and 2022?
• RQ2 How are large-scale agile development practices described and implemented in industry settings according to the literature?
• RQ3 Which studies have had the most influence in shaping the current understanding of large-scale agile development?
• RQ4 What methodological approaches are most commonly used in large-scale agile development research?
• RQ5 What are the key gaps and future research directions in the field as identified from the literature?

Key findings

• The research field has evolved significantly since 2007, with a growing number of studies focusing on organizational, technical, and process challenges in scaling agile methods.
• Common research themes include scaling frameworks (e.g., SAFe, LeSS), team coordination, tooling, and organizational transformation in large enterprises.
• A substantial portion of the studies are based on case studies or field research, indicating a strong emphasis on real-world applicability.
• The most influential studies often focus on empirical investigations of agile adoption in large organizations, particularly in the context of enterprise software development.
• There is a notable lack of longitudinal studies and standardized evaluation metrics, highlighting a gap in long-term impact assessment.
• The research agenda identifies emerging needs in areas such as agile governance, tool integration, and sustainability of agile transformations at scale.