[Paper Review] Collaborative Application Security Testing for DevSecOps: An Empirical Analysis of Challenges, Best Practices and Tool Support

Roshan Namal Rajapakse, Mansooreh Zahedi+1 more|arXiv (Cornell University)|Nov 13, 2022

Software Engineering Techniques and Practices4 citations

TL;DR

This study investigates collaborative application security testing (CoAST) in DevSecOps through thematic analysis of 48 curated webinars, identifying key challenges in tooling, role clarity, and team collaboration. It proposes actionable best practices—such as Shift-left security and ChatOps—and highlights critical tooling gaps, including lack of built-in collaboration features in AST tools, calling for enhanced integration, shared dashboards, and socio-technical metrics for future research and tool development.

ABSTRACT

DevSecOps is a software development paradigm that places a high emphasis on the culture of collaboration between developers (Dev), security (Sec) and operations (Ops) teams to deliver secure software continuously and rapidly. Adopting this paradigm effectively, therefore, requires an understanding of the challenges, best practices and available solutions for collaboration among these functional teams. However, collaborative aspects related to these teams have received very little empirical attention in the DevSecOps literature. Hence, we present a study focusing on a key security activity, Application Security Testing (AST), in which practitioners face difficulties performing collaborative work in a DevSecOps environment. Our study made novel use of 48 systematically selected webinars, technical talks and panel discussions as a data source to qualitatively analyse software practitioner discussions on the most recent trends and emerging solutions in this highly evolving field. We find that the lack of features that facilitate collaboration built into the AST tools themselves is a key tool-related challenge in DevSecOps. In addition, the lack of clarity related to role definitions, shared goals, and ownership also hinders Collaborative AST (CoAST). We also captured a range of best practices for collaboration (e.g., Shift-left security), emerging communication methods (e.g., ChatOps), and new team structures (e.g., hybrid teams) for CoAST. Finally, our study identified several requirements for new tool features and specific gap areas for future research to provide better support for CoAST in DevSecOps.

Motivation & Objective

• To investigate the collaborative challenges in Application Security Testing (AST) within DevSecOps environments, focusing on practitioner experiences.
• To identify and categorize key challenges related to team roles, tooling limitations, and communication in CoAST workflows.
• To extract and classify emerging best practices and tool capabilities that support effective collaboration in DevSecOps.
• To identify specific feature requirements and research gaps for next-generation AST tools to improve collaborative security testing.
• To provide evidence-based recommendations for tool developers and organizations to enhance CoAST through improved tooling and team practices.

Proposed method

• Systematic selection of 48 webinars from an initial pool of 3,389 YouTube videos based on relevance to DevSecOps and CoAST.
• Thematic analysis of webinar transcripts to identify recurring challenges, best practices, and tooling trends.
• Exclusion of webinars with dominant marketing content to ensure objectivity and focus on technical and collaborative insights.
• Categorization of findings into themes: challenges (nine key issues), best practices (eleven practices across five themes), and tool capabilities (ten types across six themes).
• Identification of tooling requirements through analysis of speaker recommendations and emerging trends in collaboration platforms and AST tools.
• Use of qualitative data synthesis to derive actionable recommendations for tool development and future research in CoAST.

Experimental results

Research questions

• RQ1 What are the primary challenges practitioners face in performing Collaborative Application Security Testing (CoAST) within DevSecOps environments?
• RQ2 What best practices are recommended by industry practitioners to improve collaboration in CoAST workflows?
• RQ3 Which tool features and capabilities are highlighted as essential for supporting effective CoAST in DevSecOps?
• RQ4 What gaps exist in current AST tools regarding built-in collaboration support, and what are the implications for tool development?
• RQ5 What emerging collaboration models and technologies (e.g., ChatOps, AI) are being adopted or recommended in CoAST, and what research opportunities do they present?

Key findings

• A lack of built-in collaboration features in AST tools is a major technical barrier to effective CoAST, leading to tool-switching and fragmented workflows.
• Unclear role definitions, shared goals, and ownership of security responsibilities significantly hinder collaboration between developers, security teams, and operations.
• Best practices such as Shift-left and Shift-right security, hybrid team structures, and the use of Security and DevOps champions are widely recommended to improve CoAST.
• Emerging communication methods like ChatOps and role-based access control are seen as effective for enhancing real-time collaboration and reducing friction.
• There is a strong need for centralized, context-aware dashboards that aggregate and prioritize vulnerability data across teams to improve decision-making.
• Existing AST tools often lack robust integration capabilities (e.g., APIs), and vendors are urged to improve interoperability with collaboration and communication platforms.