[论文解读] AI Chatbots as Professional Service Agents: Developing a Professional Identity

Wenwen Li, Keqiang Shi+1 more|ArXiv.org|Jan 24, 2025

Artificial Intelligence in Healthcare and Education被引用 3

一句话总结

本文提出 LAPI，一种为 AI 聊天机器人赋予专业医疗身份的框架，使用理论引导的任务规划（HBM）与务实熵方法，在医疗问答中相较基线提示提升流畅性、同理心与以患者为中心性。

ABSTRACT

With the rapid expansion of large language model (LLM) applications, there is an emerging shift in the role of LLM-based AI chatbots from serving merely as general inquiry tools to acting as professional service agents. However, current studies often overlook a critical aspect of professional service agents: the act of communicating in a manner consistent with their professional identities. This is of particular importance in the healthcare sector, where effective communication with patients is essential for achieving professional goals, such as promoting patient well-being by encouraging healthy behaviors. To bridge this gap, we propose LAPI (LLM-based Agent with a Professional Identity), a novel framework for designing professional service agent tailored for medical question-and-answer (Q\&A) services, ensuring alignment with a specific professional identity. Our method includes a theory-guided task planning process that decomposes complex professional tasks into manageable subtasks aligned with professional objectives and a pragmatic entropy method designed to generate professional and ethical responses with low uncertainty. Experiments on various LLMs show that the proposed approach outperforms baseline methods, including few-shot prompting, chain-of-thought prompting, across key metrics such as fluency, naturalness, empathy, patient-centricity, and ROUGE-L scores. Additionally, the ablation study underscores the contribution of each component to the overall effectiveness of the approach.

研究动机与目标

• Motivate the shift of AI chatbots from general inquiry tools to professional service agents in healthcare.
• Develop a domain-informed, theory-driven framework (HBM) to guide task planning toward health-behavior interventions.
• Ensure professional identity alignment and ethical, low-uncertainty responses via a pragmatic entropy optimization process.
• Demonstrate improvements over baseline prompting methods across multiple LLMs on real medical Q&A data.

提出的方法

• Propose LAPI (LLM-based Agent with a Professional Identity) combining theory-guided task planning and pragmatic entropy for professional medical Q&A.
• Use Health Belief Model (HBM) to decompose professional goals into subtasks (Perceived Threat and Perceived Effectiveness of Health Behavior).
• Assign query weights to HBM domains, generate initial prompts reflecting these domains, and apply pragmatic rules to steer responses.
• Iteratively optimize prompts via pragmatic entropy to maximize rule satisfaction and minimize response distribution uncertainty.
• Evaluate responses using both LLM-based metrics (G-EVAL for fluency, coherence, naturalness, empathy, patient-centricity) and ROUGE-L measures.

实验结果

研究问题

• RQ1 How can a professional identity be instantiated in an LLM-based chatbot for medical Q&A?
• RQ2 Does a theory-guided task planning approach improve the quality and alignment of chatbot responses with professional goals?
• RQ3 Can a pragmatic entropy method yield more professional and ethically consistent outputs than baseline prompting strategies?

主要发现

• LAPI consistently improves fluency, coherence, naturalness, empathy, and patient-centricity across several models compared with baseline prompts.
• LAPI achieves higher ROUGE-L recall and precision than baselines in medical Q&A evaluation.
• Ablation study shows both the Health Belief Model framework and Pragmatic Entropy contribute positively to performance.
• GPT-3.5-Turbo and GPT-4 benefit more from LAPI, indicating larger models better leverage the framework.