[論文レビュー] The Metacognitive Demands and Opportunities of Generative AI

Lev Tankelevitch, Viktor Kewenig+5 more|arXiv (Cornell University)|Dec 18, 2023

Explainable Artificial Intelligence (XAI)参考文献 203被引用数 17

ひとこと要約

論文は、生成AIの使いやすさの課題を理解・設計するためのメタ認知中心の視点を提案し、ユーザーに求められる点とそれに対処する設計戦略を概説します。

ABSTRACT

Generative AI (GenAI) systems offer unprecedented opportunities for transforming professional and personal work, yet present challenges around prompting, evaluating and relying on outputs, and optimizing workflows. We argue that metacognition$\unicode{x2013}$the psychological ability to monitor and control one's thoughts and behavior$\unicode{x2013}$offers a valuable lens to understand and design for these usability challenges. Drawing on research in psychology and cognitive science, and recent GenAI user studies, we illustrate how GenAI systems impose metacognitive demands on users, requiring a high degree of metacognitive monitoring and control. We propose these demands could be addressed by integrating metacognitive support strategies into GenAI systems, and by designing GenAI systems to reduce their metacognitive demand by targeting explainability and customizability. Metacognition offers a coherent framework for understanding the usability challenges posed by GenAI, and provides novel research and design directions to advance human-AI interaction.

研究の動機と目的

• Ground GenAI usability challenges in metacognition (monitoring and control of one’s thoughts and actions).
• Identify specific metacognitive demands in prompting, evaluating outputs, and automation strategy.
• Propose metacognitive interventions and design directions to improve GenAI usability.
• Suggest methods to reduce metacognitive load through explainability and customization.

提案手法

• Review of metacognition theory from psychology and cognitive science.
• Synthesis of GenAI user studies highlighting prompting, evaluation, and automation challenges.
• Proposition of two design directions: metacognitive support strategies and reduced metacognitive demand via explainability and customization.
• Integration of intervention studies and GenAI prototypes to outline research and design opportunities.

実験結果

リサーチクエスチョン

• RQ1 What metacognitive demands do users experience when prompting GenAI, evaluating outputs, and deciding on automation strategies?
• RQ2 How can metacognitive support strategies be integrated into GenAI systems to improve usability?
• RQ3 How can GenAI systems be designed to reduce metacognitive demands through explainability and customization?
• RQ4 What research directions follow from viewing GenAI usability through a metacognition framework?

主な発見

• GenAI imposes high metacognitive monitoring and control demands due to prompt formulation, output evaluation, and automation decisions.
• Metacognitive interventions can improve users’ judgment accuracy and task planning; training and feedback are effective strategies.
• Explainability and customization can offload metacognitive processing and tailor user interaction to needs.
• GenAI’s model flexibility, generality, and originality offer design opportunities to address metacognitive demands.
• There is a link between cognitive load and metacognitive demands, suggesting balance through design choices.