[Paper Review] Embrace Opportunities and Face Challenges: Using ChatGPT in Undergraduate Students' Collaborative Interdisciplinary Learning

Gaoxia Zhu, Xiuyi Fan+7 more|arXiv (Cornell University)|May 23, 2023

Artificial Intelligence in Healthcare and Education26 citations

TL;DR

A quasi-experimental study (n=130) examines how ChatGPT affects undergraduate collaborative interdisciplinary learning over two weeks, finding topic- and discipline-dependent effects on engagement and problem-solving and highlighting mixed perceptions in reflections.

ABSTRACT

ChatGPT, launched in November 2022, has gained widespread attention from students and educators globally, with an online report by Hu (2023) stating it as the fastest-growing consumer application in history. While discussions on the use of ChatGPT in higher education are abundant, empirical studies on its impact on collaborative interdisciplinary learning are rare. To investigate its potential, we conducted a quasi-experimental study with 130 undergraduate students (STEM and non-STEM) learning digital literacy with or without ChatGPT over two weeks. Weekly surveys were conducted on collaborative interdisciplinary problem-solving, physical and cognitive engagement, and individual reflections on ChatGPT use. Analysis of survey responses showed significant main effects of topics on collaborative interdisciplinary problem-solving and physical and cognitive engagement, a marginal interaction effect between disciplinary backgrounds and ChatGPT conditions for cognitive engagement, and a significant interaction effect for physical engagement. Sentiment analysis of student reflections suggested no significant difference between STEM and non-STEM students' opinions towards ChatGPT. Qualitative analysis of reflections generated eight positive themes, including efficiency, addressing knowledge gaps, and generating human-like responses, and eight negative themes, including generic responses, lack of innovation, and counterproductive to self-discipline and thinking. Our findings suggest that ChatGPT use needs to be optimized by considering the topics being taught and the disciplinary backgrounds of students rather than applying it uniformly. These findings have implications for both pedagogical research and practices.

Motivation & Objective

• Investigate the impact of ChatGPT on undergraduate collaborative interdisciplinary problem-solving.
• Examine physical and cognitive engagement changes when ChatGPT is used.
• Assess student reflections to identify perceived benefits and drawbacks of ChatGPT in interdisciplinary learning.

Proposed method

• Conduct a quasi-experimental design with 130 undergraduate students (STEM and non-STEM).
• Two-week learning period on digital literacy with or without ChatGPT.
• Weekly surveys measuring collaborative problem-solving, physical/cognitive engagement, and reflections on ChatGPT use.
• Sentiment analysis of student reflections to compare STEM vs non-STEM opinions.
• Qualitative analysis identifies themes of benefits and drawbacks.

Experimental results

Research questions

• RQ1 Does ChatGPT use affect collaborative interdisciplinary problem-solving compared to not using ChatGPT?
• RQ2 How does ChatGPT influence physical and cognitive engagement across topics and disciplines?
• RQ3 What are students’ perceived benefits and drawbacks of using ChatGPT in interdisciplinary learning?
• RQ4 Are thereDifferences in opinions toward ChatGPT between STEM and non-STEM students?

Key findings

• Significant main effects of topics on collaborative interdisciplinary problem-solving.
• Significant main effects of topics on physical and cognitive engagement.
• A marginal interaction effect between disciplinary backgrounds and ChatGPT conditions for cognitive engagement.
• A significant interaction effect for physical engagement.
• Sentiment analysis showed no significant difference in opinions toward ChatGPT between STEM and non-STEM students.
• Qualitative analysis revealed eight positive themes (e.g., efficiency, addressing knowledge gaps, human-like responses) and eight negative themes (e.g., generic responses, lack of innovation, undermining self-discipline).