[Paper Review] AI for Agile development: a Meta-Analysis

Beatriz Cabrero‐Daniel|arXiv (Cornell University)|May 14, 2023

Software Engineering Techniques and Practices4 citations

TL;DR

This meta-analysis investigates the integration of AI into Agile software development, identifying key challenges such as manual task dependency and lack of expertise, while demonstrating how AI enhances continuous integration, test prioritization, and backlog management. The study reveals that AI improves efficiency and quality but faces critical socio-technical and human-factor challenges requiring further research.

ABSTRACT

This study explores the benefits and challenges of integrating Artificial Intelligence with Agile software development methodologies, focusing on improving continuous integration and delivery. A systematic literature review and longitudinal meta-analysis of the retrieved studies was conducted to analyse the role of Artificial Intelligence and it's future applications within Agile software development. The review helped identify critical challenges, such as the need for specialised socio-technical expertise. While Artificial Intelligence holds promise for improved software development practices, further research is needed to better understand its impact on processes and practitioners, and to address the indirect challenges associated with its implementation.

Motivation & Objective

• To identify and categorize the primary challenges in Agile software development as reported in the literature.
• To analyze how AI is currently used to mitigate these Agile development challenges.
• To explore future challenges—both direct and indirect—associated with integrating AI into Agile practices.
• To highlight gaps in current research, particularly the disconnect between industry tools (e.g., JIRA) and academic literature.
• To emphasize the need for further research on human factors and socio-technical expertise in AI-Agile integration.

Proposed method

• Conducted a systematic literature review and longitudinal meta-analysis of 377 peer-reviewed, full-text English articles (6+ pages) from four academic databases and Google Scholar.
• Used keyword and Boolean queries to retrieve studies on AI and Agile development, excluding those focused solely on AI development methodologies.
• Applied thematic analysis to categorize challenges and mitigation strategies, mapping AI roles to specific Agile issues.
• Employed statistical techniques to quantitatively synthesize findings across studies, focusing on trends in AI application.
• Utilized an ad hoc web crawler (AHK) to enhance data collection from pre-print repositories and databases.
• Mapped identified challenges (n=7) to AI mitigation strategies (M1–M5), assessing AI’s role in each context.

Experimental results

Research questions

• RQ1 What are the main challenges in Agile software development as identified in the literature (RQ1)?
• RQ2 How is AI currently used to mitigate these Agile development challenges (RQ2)?
• RQ3 What are the future challenges—both direct and indirect—associated with integrating AI into Agile practices (RQ3)?
• RQ4 How do current AI tools align with real-world Agile tools like JIRA, and what gaps exist in academic coverage?
• RQ5 What socio-technical and human-factor challenges impede effective AI-Agile integration?

Key findings

• AI is increasingly used to automate and enhance core Agile practices, particularly in test case prioritization, backlog prioritization, and continuous integration.
• AI tools such as LLMs are being leveraged to assist novice developers in architectural analysis and user story evaluation, reducing reliance on expert knowledge.
• The most effective AI applications focus on mitigating human factors, such as reducing manual effort in task allocation and conflict detection in user stories.
• Despite progress, a significant gap exists between industry tools (e.g., JIRA) and academic research, with minimal mention of real-world tools in the literature.
• Critical challenges remain in AI model management, deployment, monitoring, and ensuring trustworthiness, especially in safety-critical domains.
• Security threats, including adversarial attacks on AI models and data privacy risks, are major concerns requiring integration of secure software development practices.