[论文解读] Enhancing Physics Learning with ChatGPT, Bing Chat, and Bard as Agents-to-Think-With: A Comparative Case Study

Renato P. dos Santos|arXiv (Cornell University)|Jun 1, 2023

Artificial Intelligence in Healthcare and Education被引用 8

一句话总结

一个单案例研究比较四个GenAI聊天机器人（ChatGPT-3.5、ChatGPT-4、Bing Chat 和 Bard）作为物理学习中的代理人-思考，分析与模拟学生的交互日志以评估同理心、学科知识与学习过程理解。

ABSTRACT

The rise of AI has brought remarkable advancements in education, with AI models demonstrating their ability to analyse and provide instructive solutions to complex problems. This study compared and analysed the responses of four Generative AI-powered chatbots (GenAIbots) - ChatGPT-3.5, ChatGPT-4, Bing Chat, and Bard - within the constructivist theoretical framework. Using a single-case study methodology, interaction logs between the GenAIbots and a simulated student in Physics learning scenarios were analysed. The GenAIbots were presented with conceptually dense Physics problems to promote deep understanding. The qualitative analysis focused on tutor traits such as subject-matter knowledge, empathy, assessment emphasis, facilitation skills, and comprehension of the learning process. Findings showed that all GenAIbots functioned as agents-to-think-with, fostering critical thinking, problem-solving, and subject-matter knowledge. ChatGPT-4 stood out for demonstrating empathy and a deep understanding of the learning process. However, inconsistencies and shortcomings were observed, highlighting the need for human intervention in AI-assisted learning. In conclusion, while GenAIbots have limitations, their potential as agents-to-think-with in Physics education offers promising prospects for revolutionising instruction.

研究动机与目标

• Investigate how GenAI-powered chatbots function as agents-to-think-with in physics learning within a constructivist framework.
• Analyse interaction logs between four GenAIbots and a simulated physics student to assess tutoring traits.
• Identify strengths and limitations of each chatbot in promoting critical thinking, problem solving, and subject-matter knowledge.
• Highlight implications for human intervention and the potential of GenAIbots to transform physics instruction.

提出的方法

• Single-case study using interaction logs between GenAIbots and a simulated student in physics learning scenarios.
• Qualitative analysis focusing on tutor traits: subject-matter knowledge, empathy, assessment emphasis, facilitation skills, and understanding of the learning process.
• Comparison across four GenAIbots: ChatGPT-3.5, ChatGPT-4, Bing Chat, and Bard.
• Assessment of their ability to promote deep understanding through conceptually dense physics problems.

实验结果

研究问题

• RQ1 How do ChatGPT-3.5, ChatGPT-4, Bing Chat, and Bard function as agents-to-think-with in physics learning?
• RQ2 Which tutor traits (knowledge, empathy, assessment emphasis, facilitation, learning-process comprehension) are exhibited by each GenAIbot?
• RQ3 What are the strengths and limitations of each GenAIbot in fostering critical thinking and problem solving in physics?
• RQ4 To what extent is human intervention needed to support AI-assisted physics instruction?

主要发现

• All GenAIbots functioned as agents-to-think-with, fostering critical thinking, problem-solving, and subject-matter knowledge.
• ChatGPT-4 stood out for demonstrating empathy and a deep understanding of the learning process.
• Inconsistencies and shortcomings were observed across bots, indicating the need for human oversight in AI-assisted learning.
• GenAIbots show potential to revolutionize physics instruction, but require consideration of limitations and appropriate human support.