[Paper Review] Ethical Aspects of ChatGPT in Software Engineering Research

Muhammad Azeem Akbar, Arif Ali Khan+1 more|arXiv (Cornell University)|Jun 13, 2023

Artificial Intelligence in Healthcare and Education11 citations

TL;DR

The paper identifies motivators, demotivators, and ethical principles for using ChatGPT in software engineering (SE) research, validates them via a questionnaire, and develops ISM and MICMAC-based decision models to guide ethical integration. It reports survey-backed insights from 113 respondents and an ISM input from 23 experts.

ABSTRACT

ChatGPT can improve Software Engineering (SE) research practices by offering efficient, accessible information analysis and synthesis based on natural language interactions. However, ChatGPT could bring ethical challenges, encompassing plagiarism, privacy, data security, and the risk of generating biased or potentially detrimental data. This research aims to fill the given gap by elaborating on the key elements: motivators, demotivators, and ethical principles of using ChatGPT in SE research. To achieve this objective, we conducted a literature survey, identified the mentioned elements, and presented their relationships by developing a taxonomy. Further, the identified literature-based elements (motivators, demotivators, and ethical principles) were empirically evaluated by conducting a comprehensive questionnaire-based survey involving SE researchers. Additionally, we employed Interpretive Structure Modeling (ISM) approach to analyze the relationships between the ethical principles of using ChatGPT in SE research and develop a level based decision model. We further conducted a Cross-Impact Matrix Multiplication Applied to Classification (MICMAC) analysis to create a cluster-based decision model. These models aim to help SE researchers devise effective strategies for ethically integrating ChatGPT into SE research by following the identified principles through adopting the motivators and addressing the demotivators. The findings of this study will establish a benchmark for incorporating ChatGPT services in SE research with an emphasis on ethical considerations.

Motivation & Objective

• Identify motivators and demotivators affecting ethical use of ChatGPT in SE research.
• Define and categorize ethical principles guiding ChatGPT deployment in SE research.
• Validate literature-derived elements through a questionnaire with SE researchers.
• Model relationships among ethical principles using Interpretive Structure Modeling (ISM).
• Develop MICMAC-based cluster decision models to aid ethical strategy in SE research.

Proposed method

• Conduct a literature survey (including grey literature) to extract motivators, demotivators, and ethical principles.
• Validate findings with a questionnaire using a five-point Likert scale (n=113 respondents from 19 countries).
• Apply Interpretive Structure Modeling (ISM) to derive a multi-level relationship map among principles (involving 23 experts).
• Execute a Cross-Impact Matrix Multiplication Applied to Classification (MICMAC) analysis for a cluster-based decision framework.
• Report implications and provide guidelines for ethically integrating ChatGPT into SE research.

Figure 1: Proposed Research Methodology

Experimental results

Research questions

• RQ1 What motivators encourage the use of ChatGPT in SE research, and what demotivators hinder it?
• RQ2 What ethical principles govern the use of ChatGPT in SE research, and how are they related?
• RQ3 How do motivators, demotivators, and principles interact, and what decision models can guide ethical adoption?
• RQ4 What is the structure of relationships among ethical principles when applying ChatGPT in SE research?
• RQ5 How can ISM and MICMAC be used to support ethical decision-making in SE research with ChatGPT?

Key findings

• 14 identified motivators (e.g., synthetic data generation, domain-specific fine-tuning, usability simulation) support ethical adoption in SE research.
• Demotivators (e.g., model limitations, data-intensive fine-tuning, convention misalignment) may undermine ethical use and need mitigation.
• Empirical survey (n=113, from 19 countries) shows ~80-86% agreement on the importance of identified ethical principles (notably Bias, Privacy, and Fairness).
• Bias (P1), Privacy (P2), and Fairness (P14) are considered highly important ethical principles by about 85-86% of respondents.
• Manipulation (P7) is perceived as a less significant principle than bias, privacy, and fairness (about 72%).
• ISM reveals a five-level structure showing interdependencies among ethical principles; MICMAC supports a cluster-based decision view for ethical integration.

Figure 2: Mapping of motivators and demotivators against research ethics