[Paper Review] Personalizing Prostate Cancer Education for Patients Using an EHR-Integrated LLM Agent

Yuexing Hao, Jason Holmes+11 more|arXiv (Cornell University)|Sep 27, 2024

Artificial Intelligence in Healthcare and EducationMedicine3 citations

TL;DR

This paper presents MedEduChat, an EHR-integrated, expert-in-the-loop LLM-powered chatbot that delivers personalized, semi-structured prostate cancer education by leveraging patient-specific electronic health record data. Co-designed with patients, clinicians, and AI experts, the chatbot improves accessibility and patient engagement in a usability study, demonstrating the feasibility of LLMs in patient-centered oncology education with strong human-AI collaboration principles.

ABSTRACT

Cancer patients often lack timely education and personalized support due to clinician workload. This quality improvement study develops and evaluates a Large Language Model (LLM) agent, MedEduChat, which is integrated with the clinic's electronic health records (EHR) and designed to enhance prostate cancer patient education. Fifteen non-metastatic prostate cancer patients and three clinicians recruited from the Mayo Clinic interacted with the agent between May 2024 and April 2025. Findings showed that MedEduChat has a high usability score (UMUX 83.7 out of 100) and improves patients' health confidence (Health Confidence Score rose from 9.9 to 13.9). Clinicians evaluated the patient-chat interaction history and rated MedEduChat as highly correct (2.9 out of 3), complete (2.7 out of 3), and safe (2.7 out of 3), with moderate personalization (2.3 out of 3). This study highlights the potential of LLM agents to improve patient engagement and health education.

Motivation & Objective

• Address the critical gap in timely, personalized patient education for prostate cancer patients due to limited clinical time and low health literacy.
• Develop a scalable, secure, and patient-centered LLM-based chatbot that integrates with electronic health records to deliver real-time, tailored education.
• Establish a co-design framework involving patients, clinicians, and AI researchers to ensure clinical relevance, usability, and ethical alignment.
• Evaluate the chatbot’s effectiveness in enhancing patient understanding and engagement through usability testing with real patients and clinicians.
• Define the practical boundaries and design principles for deploying LLMs in clinical patient education, emphasizing safety, accuracy, and accessibility.

Proposed method

• Conducted a needs-assessment survey with prostate cancer patients using existing e-learning modules to identify key educational gaps and pain points.
• Employed a co-design process involving patients, clinicians, and AI experts to iteratively shape the chatbot’s structure, tone, and functionality.
• Built MedEudChat on a closed-domain, fine-tuned LLM (ChatGPT-4) to ensure accuracy and safety by restricting responses to prostate cancer-specific, evidence-based content.
• Integrated the chatbot with electronic health records (EHRs) to enable personalized responses based on patient-specific data such as diagnosis stage and treatment plan.
• Designed semi-structured, conversational interactions to guide patients through key educational topics in a non-overwhelming, step-by-step manner.
• Conducted a usability study with seven patients and three clinicians to evaluate response quality, clarity, and perceived helpfulness using qualitative feedback and expert review.

Experimental results

Research questions

• RQ1 What are the primary challenges in current prostate cancer patient education, particularly regarding accessibility, personalization, and information overload?
• RQ2 How can a co-design process involving patients, clinicians, and AI experts lead to a more effective and trustworthy LLM-based educational tool?
• RQ3 To what extent does the LLM-powered chatbot improve patient understanding and engagement in prostate cancer education compared to standard e-learning modules?
• RQ4 What are the practical and ethical boundaries of deploying LLMs in clinical patient education, especially in terms of accuracy, bias mitigation, and patient safety?

Key findings

• Patients reported that MedEduChat provided clearer, more personalized explanations than standard e-learning modules, particularly regarding treatment options and side effects.
• Clinicians found the chatbot’s responses to be clinically accurate, well-structured, and aligned with current guidelines, with minimal hallucination or misinformation.
• The co-design process successfully identified key usability and safety requirements, such as avoiding jargon, providing context-specific answers, and ensuring emotional sensitivity.
• The integration with EHR data enabled the chatbot to deliver tailored responses based on patient-specific factors like treatment stage and comorbidities.
• Despite limitations in sample diversity, patients with varying educational backgrounds found the chatbot accessible and engaging, suggesting potential for broad applicability.
• The study highlighted the need for future research on cognitive load, language accessibility, and usability among non-English speakers and patients with low health literacy.