[论文解读] Enhancing Critical Thinking in Generative AI Search with Metacognitive Prompts

Anjali Singh, Zhitong Guan+1 more|ArXiv.org|May 29, 2025

AI in Service Interactions被引用 3

一句话总结

该论文研究元认知提示是否能够在基于 GenAI 的搜索中提升批判性思维与参与度，采用面向大学生的用户研究。

ABSTRACT

The growing use of Generative AI (GenAI) conversational search tools has raised concerns about their effects on people's metacognitive engagement, critical thinking, and learning. As people increasingly rely on GenAI to perform tasks such as analyzing and applying information, they may become less actively engaged in thinking and learning. This study examines whether metacognitive prompts - designed to encourage people to pause, reflect, assess their understanding, and consider multiple perspectives - can support critical thinking during GenAI-based search. We conducted a user study (N=40) with university students to investigate the impact of metacognitive prompts on their thought processes and search behaviors while searching with a GenAI tool. We found that these prompts led to more active engagement, prompting students to explore a broader range of topics and engage in deeper inquiry through follow-up queries. Students reported that the prompts were especially helpful for considering overlooked perspectives, promoting evaluation of AI responses, and identifying key takeaways. Additionally, the effectiveness of these prompts was influenced by students' metacognitive flexibility. Our findings highlight the potential of metacognitive prompts to foster critical thinking and provide insights for designing and implementing metacognitive support in human-AI interactions.

研究动机与目标

• 在使用 GenAI 搜索工具时，推动保留并培养元认知参与与批判性思维的需求。
• 检验元认知提示是否影响 GenAI 辅助搜索过程中的思维过程与搜索行为。
• 评估提示对主题探索深度、探究深度以及对多元观点的考虑的影响。
• 评估用户对用于评估 AI 输出与提炼要点的提示有用性的自我报告。

提出的方法

• 对大学生进行受控用户研究（N=40）。
• 在有无元认知提示的条件下，让参与者使用基于 GenAI 的搜索工具。
• 通过参与者报告与交互记录捕捉思维过程与搜索行为。
• 分析提示是否促使更广泛的主题探索与更深入的后续查询。
• 评估提示在考虑被忽略的观点与评估 AI 回应方面的感知有用性。
• 检验元认知灵活性的个体差异如何调节提示的效力。

实验结果

研究问题

• RQ1 元认知提示是否在 GenAI 搜索中提高主动参与度和更广的主题探索？
• RQ2 提示是否促进更深入的探究以及对 AI 生成信息的更有效评估？
• RQ3 元认知灵活性的个体差异如何影响提示的有效性？
• RQ4 提示是否帮助用户考虑被忽略的观点并从 AI 回应中提取要点？

主要发现

• 提示导致更积极的参与和对主题的更广泛探索。
• 提示通过后续查询促使更深入的探究。
• 参与者表示提示有助于考虑被忽略的观点并评估 AI 回应。
• 提示在识别 AI 输出的要点方面尤其有帮助。
• 提示的有效性受到用户元认知灵活性的影响。