[Paper Review] Natural Language Programming in Medicine: Administering Evidence Based Clinical Workflows with Autonomous Agents Powered by Generative Large Language Models

Akhil Vaid, Joshua Lampert+28 more|arXiv (Cornell University)|Jan 5, 2024

Artificial Intelligence in Healthcare and Education7 citations

TL;DR

The paper evaluates autonomous agents powered by generative LLMs to administer evidence-based clinical workflows in a simulated tertiary care setting, comparing proprietary and open-source models with RAG, and highlighting the need for human oversight and NLP-based behavior modification.

ABSTRACT

Generative Large Language Models (LLMs) hold significant promise in healthcare, demonstrating capabilities such as passing medical licensing exams and providing clinical knowledge. However, their current use as information retrieval tools is limited by challenges like data staleness, resource demands, and occasional generation of incorrect information. This study assessed the potential of LLMs to function as autonomous agents in a simulated tertiary care medical center, using real-world clinical cases across multiple specialties. Both proprietary and open-source LLMs were evaluated, with Retrieval Augmented Generation (RAG) enhancing contextual relevance. Proprietary models, particularly GPT-4, generally outperformed open-source models, showing improved guideline adherence and more accurate responses with RAG. The manual evaluation by expert clinicians was crucial in validating models' outputs, underscoring the importance of human oversight in LLM operation. Further, the study emphasizes Natural Language Programming (NLP) as the appropriate paradigm for modifying model behavior, allowing for precise adjustments through tailored prompts and real-world interactions. This approach highlights the potential of LLMs to significantly enhance and supplement clinical decision-making, while also emphasizing the value of continuous expert involvement and the flexibility of NLP to ensure their reliability and effectiveness in healthcare settings.

Motivation & Objective

• Motivate and assess the use of autonomous LLM agents to execute evidence-based clinical workflows in medicine.
• Compare proprietary and open-source LLMs in terms of guideline adherence and response accuracy within a tertiary care simulation.
• Assess the impact of Retrieval Augmented Generation (RAG) on contextual relevance and decision quality.
• Demonstrate Natural Language Programming as a practical paradigm for safely adjusting model behavior in clinical contexts.

Proposed method

• Simulate a tertiary care medical center using real-world clinical cases across multiple specialties.
• Evaluate both proprietary and open-source LLMs for autonomous clinical task execution.
• Incorporate Retrieval Augmented Generation (RAG) to enhance contextual relevance of outputs.
• Apply expert clinician manual evaluation to validate model outputs.
• Advocate Natural Language Programming (NLP) as a paradigm for modifying model behavior via prompts and real-world interactions.

Experimental results

Research questions

• RQ1 Can autonomous LLM agents reliably adhere to clinical guidelines across multiple specialties in a simulated hospital setting?
• RQ2 Do proprietary models (e.g., GPT-4) outperform open-source models in guideline adherence and accuracy when using RAG?
• RQ3 Does Retrieval Augmented Generation improve the contextual relevance and correctness of LLM-driven clinical workflows?
• RQ4 What is the role of human expert oversight in validating and supervising autonomous medical agents?
• RQ5 Is Natural Language Programming a viable and effective method to tune autonomous clinical agents for reliability and safety?

Key findings

• Proprietary models, particularly GPT-4, generally outperform open-source models in guideline adherence and accuracy when using RAG.
• RAG enhances contextual relevance of responses in the medical autonomy setting.
• Manual expert clinician evaluation is crucial for validating model outputs and ensuring safe operation.
• NL Programming enables precise adjustments to model behavior through tailored prompts and interactions.
• The approach demonstrates potential for LLMs to augment clinical decision-making while requiring ongoing expert involvement.