[Paper Review] Relating IS Developers' Attitudes to Engagement

Sherlock A. Licorish, Stephen G. MacDonell|arXiv (Cornell University)|Feb 12, 2021

Software Engineering Techniques and Practices30 references3 citations

TL;DR

This study investigates how IS developers' attitudes—measured through linguistic analysis of code-related communications—relate to their engagement levels. Using text mining and statistical analysis on 474 developers' artefacts, it finds that expressions of work/achievement and insightful attitudes strongly predict higher engagement, with more communicative developers showing greater involvement.

ABSTRACT

Increasing effort is being directed to understanding the personality profiles of highly engaged information systems (IS) developers and the impact of such profiles on development outcomes. However, there has been a lesser degree of attention paid to studying attitudes at a fine-grained level, and relating such attitudes to developers' in-process activities, in spite of the fact that social motivation theory notes the importance of such a relationship in general group work. We have therefore applied linguistic analysis, text mining and visualization, and statistical analysis techniques to artefacts developed by 474 developers to study these issues. Our results indicate that our sample of IS developers conveyed a range of attitudes while working to deliver systems features, and those practitioners who communicated the most were also the most engaged. Additionally, of eight linguistic dimensions considered, expressions regarding work and achievement, as well as insightful attitudes, were most closely related to developers' engagement. Accordingly, team diversity and the provision of active support for outcome-driven developers may contribute positively to maintaining team balance and performance.

Motivation & Objective

• To examine the relationship between IS developers' fine-grained attitudes and their engagement in software development.
• To identify which linguistic dimensions of communication correlate most strongly with developer engagement.
• To understand how team diversity and support for outcome-driven developers can enhance team performance and balance.
• To apply linguistic analysis and text mining techniques to real-world development artefacts to uncover behavioral patterns linked to engagement.

Proposed method

• Conducted linguistic analysis on textual artefacts (e.g., commit messages, documentation) from 474 IS developers.
• Employed text mining techniques to extract and categorize linguistic dimensions such as work/achievement, insight, and emotional tone.
• Used statistical analysis to correlate linguistic expressions with self-reported or observable engagement levels.
• Applied data visualization to interpret patterns in communication and engagement across the developer sample.
• Classified linguistic content into eight predefined dimensions to assess their relationship with engagement.
• Validated findings using a conference paper format and peer-reviewed methodology from ACIS 2014.

Experimental results

Research questions

• RQ1 Which linguistic dimensions of developers' communications are most strongly associated with higher engagement?
• RQ2 How does the volume of communication relate to developers' levels of engagement?
• RQ3 To what extent do expressions of work and achievement correlate with engagement in IS development?
• RQ4 How do insightful attitudes in developer communications influence engagement outcomes?
• RQ5 What role does team diversity play in moderating the relationship between attitudes and engagement?

Key findings

• Developers who communicated most frequently in their artefacts demonstrated the highest levels of engagement.
• Expressions related to work and achievement showed the strongest positive correlation with engagement among the eight linguistic dimensions studied.
• Insightful attitudes—such as reflective or problem-solving language—were significantly linked to higher engagement levels.
• The study identified that team diversity and active support for outcome-driven developers may help maintain team balance and performance.
• Linguistic analysis revealed a range of nuanced attitudes expressed during feature delivery, indicating rich emotional and cognitive engagement.
• Statistical modeling confirmed that specific linguistic patterns are reliable indicators of engagement, even when controlling for communication volume.