[Paper Review] AI for DevSecOps: A Landscape and Future Opportunities

Michael C. Fu, Jirat Pasuksmit+1 more|arXiv (Cornell University)|Apr 7, 2024

Scientific Computing and Data Management4 citations

TL;DR

This paper presents a comprehensive landscape of AI-driven security techniques in DevSecOps, analyzing 99 studies (2017–2023) across 12 security tasks. It identifies 65 benchmarks, 15 key challenges in current research, and proposes 15 future opportunities to enhance security, trust, and efficiency in automated software development pipelines using AI.

ABSTRACT

DevOps has emerged as one of the most rapidly evolving software development paradigms. With the growing concerns surrounding security in software systems, the DevSecOps paradigm has gained prominence, urging practitioners to incorporate security practices seamlessly into the DevOps workflow. However, integrating security into the DevOps workflow can impact agility and impede delivery speed. Recently, the advancement of artificial intelligence (AI) has revolutionized automation in various software domains, including software security. AI-driven security approaches, particularly those leveraging machine learning or deep learning, hold promise in automating security workflows. They reduce manual efforts, which can be integrated into DevOps to ensure uninterrupted delivery speed and align with the DevSecOps paradigm simultaneously. This paper seeks to contribute to the critical intersection of AI and DevSecOps by presenting a comprehensive landscape of AI-driven security techniques applicable to DevOps and identifying avenues for enhancing security, trust, and efficiency in software development processes. We analyzed 99 research papers spanning from 2017 to 2023. Specifically, we address two key research questions (RQs). In RQ1, we identified 12 security tasks associated with the DevSecOps process and reviewed existing AI-driven security approaches, the problems they addressed, and the 65 benchmarks used to evaluate those approaches. Drawing insights from our findings, in RQ2, we discussed state-of-the-art AI-driven security approaches, highlighted 15 challenges in existing research, and proposed 15 corresponding avenues for future opportunities.

Motivation & Objective

• To map the current state of AI-driven security techniques in DevSecOps across key software development phases.
• To identify gaps and challenges in existing AI-based approaches for integrating security into DevOps workflows.
• To propose actionable future research directions that enhance automation, trust, and performance in secure software delivery.
• To evaluate the effectiveness of AI techniques using 65 standardized benchmarks across diverse security tasks.
• To bridge the gap between AI automation and DevSecOps agility by addressing practical limitations in current research.

Proposed method

• Systematic literature review of 99 peer-reviewed papers published between 2017 and 2023 in software engineering and AI venues.
• Categorization of AI-driven security techniques by DevSecOps phase, including requirements, development, testing, deployment, and monitoring.
• Identification and classification of 65 evaluation benchmarks used in the reviewed studies to assess model performance.
• Thematic analysis of challenges in current AI for DevSecOps, focusing on data quality, model interpretability, scalability, and toolchain integration.
• Synthesis of state-of-the-art approaches using machine learning and deep learning to automate security tasks such as vulnerability detection and configuration analysis.
• Proposal of 15 future research opportunities based on identified challenges, emphasizing practical deployment and cross-tool interoperability.

Experimental results

Research questions

• RQ1 What are the 12 key security tasks in the DevSecOps lifecycle that are currently being addressed by AI-driven approaches?
• RQ2 What are the most commonly used benchmarks and evaluation metrics in AI-based DevSecOps research, and how do they reflect real-world applicability?
• RQ3 What are the major challenges limiting the adoption and effectiveness of AI in DevSecOps, and how can they be addressed?
• RQ4 What are the most promising future research directions for advancing AI in DevSecOps, particularly in improving trust, scalability, and integration with CI/CD pipelines?
• RQ5 How do existing AI techniques compare in performance and generalizability across different DevSecOps stages and threat types?

Key findings

• The study identified 12 distinct security tasks within the DevSecOps lifecycle where AI is actively applied, including static analysis, dynamic scanning, and configuration hardening.
• A total of 65 unique benchmarks were used across the reviewed studies, with the majority focusing on vulnerability detection in source code and container images.
• Despite high performance on benchmarks, many AI models show limited generalizability across different programming languages and development environments.
• Key challenges include lack of explainability in AI decisions, poor integration with existing CI/CD tools, and insufficient real-world validation in production pipelines.
• There is a significant gap in research on AI for runtime security monitoring and anomaly detection in cloud-native environments.
• The authors propose 15 future research opportunities, including improving model interpretability, enhancing data efficiency, and developing standardized evaluation frameworks for AI in DevSecOps.