[Paper Review] AI Tool Use and Adoption in Software Development by Individuals and Organizations: A Grounded Theory Study

Ze Shi Li, Nowshin Nawar Arony+3 more|arXiv (Cornell University)|Jun 25, 2024

Impact of AI and Big Data on Business and SocietyDecision Sciences3 citations

TL;DR

This grounded theory study investigates individual and organizational motives and challenges influencing AI tool adoption in software development. Through 26 interviews and a survey of 395 practitioners, it identifies 2 individual and 3 organizational motives, 4 individual and 3 organizational challenges, and 3 push-pull relationships that shape adoption—offering actionable insights for improving AI tool integration in engineering teams.

ABSTRACT

AI assistance tools such as ChatGPT, Copilot, and Gemini have dramatically impacted the nature of software development in recent years. Numerous studies have studied the positive benefits that practitioners have achieved from using these tools in their work. While there is a growing body of knowledge regarding the usability aspects of leveraging AI tools, we still lack concrete details on the issues that organizations and practitioners need to consider should they want to explore increasing adoption or use of AI tools. In this study, we conducted a mixed methods study involving interviews with 26 industry practitioners and 395 survey respondents. We found that there are several motives and challenges that impact individuals and organizations and developed a theory of AI Tool Adoption. For example, we found creating a culture of sharing of AI best practices and tips as a key motive for practitioners' adopting and using AI tools. In total, we identified 2 individual motives, 4 individual challenges, 3 organizational motives, and 3 organizational challenges, and 3 interleaved relationships. The 3 interleaved relationships act in a push-pull manner where motives pull practitioners to increase the use of AI tools and challenges push practitioners away from using AI tools.

Motivation & Objective

• To understand the factors influencing individual and organizational adoption of AI tools in software development.
• To identify specific motives driving practitioners and organizations to adopt AI tools like Copilot and ChatGPT.
• To uncover challenges that hinder effective and safe use of AI tools in software engineering workflows.
• To develop a grounded theory of AI tool adoption that captures the interplay between individual and organizational factors.
• To provide evidence-based recommendations for organizations to improve AI tool adoption through training, guidelines, and culture

Proposed method

• Conducted semi-structured interviews with 26 software practitioners across diverse roles, organizations, and geographic locations.
• Applied socio-technical grounded theory (STGT) to iteratively code, memo, and compare data until theoretical saturation was reached.
• Validated theoretical categories and relationships using a survey of 395 software professionals to ensure reliability and generalizability.
• Used Otter.ai for interview transcription and SurveyMonkey for data collection, ensuring transparency and reproducibility.
• Employed the total reliability framework (credibility, analyzability, transparency, usefulness) to assess validity and mitigate research threats.
• Released a replication package with interview questions, codebook, and survey instruments to support transparency and reproducibility

Experimental results

Research questions

• RQ1 What individual and organizational motives drive the adoption of AI tools in software development?
• RQ2 What individual and organizational challenges hinder the effective use and adoption of AI tools in software engineering?
• RQ3 How do motives and challenges interact in a push-pull dynamic that influences AI tool usage?
• RQ4 What role do organizational policies, training, and culture play in shaping practitioner adoption and usage of AI tools?

Key findings

• Practitioners are primarily motivated to use AI tools by the desire to improve productivity and the opportunity to share best practices and tips within teams.
• Organizations that provide structured AI training and clear usage guidelines significantly increase practitioner adoption and confidence in using AI tools.
• Key individual challenges include lack of training, uncertainty about prompt effectiveness, concerns about code quality, and privacy risks related to data leakage.
• Organizational challenges include absence of formal AI policies, lack of ethical guidelines, and insufficient infrastructure to support secure AI usage.
• Three push-pull relationships were identified: (1) motivation to improve productivity vs. fear of code quality issues, (2) desire to share knowledge vs. concerns about intellectual property, and (3) organizational support vs. lack of governance.
• The study found that 395 survey respondents confirmed the theoretical categories, validating that the identified motives and challenges are representative of broader practitioner experiences