[Paper Review] Navigating the Complexity of Generative AI Adoption in Software Engineering

Daniel Russo|arXiv (Cornell University)|Jul 12, 2023

Big Data and Business IntelligenceBusiness, Management and Accounting18 citations

TL;DR

The paper uses a convergent mixed-methods design to model how Generative AI/LLMs are adopted in software engineering, revealing that workflow compatibility principally drives adoption at an early stage, with usefulness and social factors playing smaller roles.

ABSTRACT

In this paper, the adoption patterns of Generative Artificial Intelligence (AI) tools within software engineering are investigated. Influencing factors at the individual, technological, and societal levels are analyzed using a mixed-methods approach for an extensive comprehension of AI adoption. An initial structured interview was conducted with 100 software engineers, employing the Technology Acceptance Model (TAM), the Diffusion of Innovations theory (DOI), and the Social Cognitive Theory (SCT) as guiding theories. A theoretical model named the Human-AI Collaboration and Adaptation Framework (HACAF) was deduced using the Gioia Methodology, characterizing AI adoption in software engineering. This model's validity was subsequently tested through Partial Least Squares - Structural Equation Modeling (PLS-SEM), using data collected from 183 software professionals. The results indicate that the adoption of AI tools in these early integration stages is primarily driven by their compatibility with existing development workflows. This finding counters the traditional theories of technology acceptance. Contrary to expectations, the influence of perceived usefulness, social aspects, and personal innovativeness on adoption appeared to be less significant. This paper yields significant insights for the design of future AI tools and supplies a structure for devising effective strategies for organizational implementation.

Motivation & Objective

• Investigate what factors influence the adoption of Generative AI tools in software engineering.
• Integrate individual, technological, and social perspectives using established theories (TAM, DOI, SCT).
• Develop and validate a theoretical model (HACAF) of human–AI collaboration and adaptation in software engineering.
• Provide design and organizational guidance for effective AI tool implementation in software teams.

Proposed method

• Apply a convergent mixed-methods approach combining quantitative surveys and qualitative analysis.
• Ground the study in Technology Acceptance Model (TAM), Diffusion of Innovation (DOI), and Social Cognitive Theory (SCT).
• Use Gioia methodology for theory induction from qualitative data and PLS-SEM for model validation.
• Collect data from 100 software engineers (survey) and 183 software engineers (PLS-SEM validation).
• Conduct thematic analysis and iterative coding to derive first-order concepts, second-order themes, and aggregate dimensions.

Experimental results

Research questions

• RQ1 What influences the adoption of Generative AI tools in software engineering?

Key findings

• Adoption is predominantly driven by compatibility of AI tools with existing development workflows.
• Perceived usefulness, social factors, and personal innovativeness have a weaker impact at this early adoption stage.
• A theoretical model—Human-AI Collaboration and Adaptation Framework (HACAF)—was developed and validated.
• The findings offer guidance for AI tool design and organizational implementation strategies in software engineering.