[Paper Review] When Generative AI Meets Workplace Learning: Creating A Realistic & Motivating Learning Experience With A Generative PCA

Andreas Bucher, Birgit Schenk+2 more|arXiv (Cornell University)|May 24, 2024

Intelligent Tutoring Systems and Adaptive Learning5 citations

TL;DR

The paper proposes a Generative PCA (GenPCA) for organizational communication training and reports positive user reception and enhanced self-determined learning. It discusses limitations and improvements for workplace learning with GenPCA.

ABSTRACT

Workplace learning is used to train employees systematically, e.g., via e-learning or in 1:1 training. However, this is often deemed ineffective and costly. Whereas pure e-learning lacks the possibility of conversational exercise and personal contact, 1:1 training with human instructors involves a high level of personnel and organizational costs. Hence, pedagogical conversational agents (PCAs), based on generative AI, seem to compensate for the disadvantages of both forms. Following Action Design Research, this paper describes an organizational communication training with a Generative PCA (GenPCA). The evaluation shows promising results: the agent was perceived positively among employees and contributed to an improvement in self-determined learning. However, the integration of such agent comes not without limitations. We conclude with suggestions concerning the didactical methods, which are supported by a GenPCA, and possible improvements of such an agent for workplace learning.

Motivation & Objective

• Motivate and explore the use of a Generative PCA to enhance workplace learning through pedagogical conversational agents.
• Investigate whether GenPCA can combine the benefits of e-learning and human-led training while reducing costs.
• Evaluate user perceptions and effects on self-determined learning in an organizational training context.
• Identify limitations and practical recommendations for integrating GenPCA into workplace learning practices.

Proposed method

• Apply Action Design Research to develop and implement a GenPCA-based organizational communication training.
• Use a Generative AI-based conversational agent to support pedagogical interactions and practice.
• Assess user perceptions and learning autonomy outcomes through qualitative and/or quantitative evaluation.
• Analyze limitations and provide didactical recommendations with concrete improvement suggestions.

Experimental results

Research questions

• RQ1 Does a Generative PCA support a realistic and motivating learning experience in workplace training?
• RQ2 What is the impact of GenPCA on employees' self-determined learning and engagement?
• RQ3 What limitations arise from using GenPCA for organizational training, and how can they be mitigated?
• RQ4 What didactical methods are best supported or enhanced by GenPCA in workplace learning?

Key findings

• The GenPCA-based agent was perceived positively among employees.
• The agent contributed to an improvement in self-determined learning.
• There are integration limitations to consider when applying GenPCA to workplace learning.
• The paper provides didactical recommendations and potential improvements for GenPCA in such settings.