[Paper Review] Polaris: A Safety-focused LLM Constellation Architecture for Healthcare

Subhabrata Mukherjee, Paul Gamble+24 more|arXiv (Cornell University)|Mar 20, 2024

Quality and Safety in Healthcare19 citations

TL;DR

Polaris presents a safety-focused, multi-agent LLM constellation for real-time, patient-facing healthcare conversations, with a primary agent and specialist support agents to enhance medical safety and reduce hallucinations.

ABSTRACT

We develop Polaris, the first safety-focused LLM constellation for real-time patient-AI healthcare conversations. Unlike prior LLM works in healthcare focusing on tasks like question answering, our work specifically focuses on long multi-turn voice conversations. Our one-trillion parameter constellation system is composed of several multibillion parameter LLMs as co-operative agents: a stateful primary agent that focuses on driving an engaging conversation and several specialist support agents focused on healthcare tasks performed by nurses to increase safety and reduce hallucinations. We develop a sophisticated training protocol for iterative co-training of the agents that optimize for diverse objectives. We train our models on proprietary data, clinical care plans, healthcare regulatory documents, medical manuals, and other medical reasoning documents. We align our models to speak like medical professionals, using organic healthcare conversations and simulated ones between patient actors and experienced nurses. This allows our system to express unique capabilities such as rapport building, trust building, empathy and bedside manner. Finally, we present the first comprehensive clinician evaluation of an LLM system for healthcare. We recruited over 1100 U.S. licensed nurses and over 130 U.S. licensed physicians to perform end-to-end conversational evaluations of our system by posing as patients and rating the system on several measures. We demonstrate Polaris performs on par with human nurses on aggregate across dimensions such as medical safety, clinical readiness, conversational quality, and bedside manner. Additionally, we conduct a challenging task-based evaluation of the individual specialist support agents, where we demonstrate our LLM agents significantly outperform a much larger general-purpose LLM (GPT-4) as well as from its own medium-size class (LLaMA-2 70B).

Motivation & Objective

• Develop a real-time, patient-facing healthcare conversation system emphasizing safety and non-diagnostic support tasks.
• Reduce hallucinations and increase medical accuracy via a multi-agent constellation with specialized support models.
• Enable nurse-like rapport, empathy, and bedside manner in AI-driven conversations.
• Maintain conversation state over long, multi-turn interactions and handle voice-based communication.
• Evaluate system performance against human nurses and compare specialist agents to general LLMs.

Proposed method

• Design a multi-agent LLM constellation with a stateful primary agent and several specialist support agents.
• Train the primary agent with general instruction tuning, conversation/agent tuning, and RLHF using proprietary medical data and simulated conversations.
• Develop a message-passing orchestration framework that updates conversation state via outputs from specialist agents.
• Use dedicated specialist agents (privacy, medication, labs/vitals, nutrition, policy, EHR, human intervention, etc.) to provide context and safety checks.
• Apply model and data strategies including 50-100B parameter support models, bf16/int8 precision, and performance optimizations (GQA, Flash Attention 2, RoPE).
• Employ simulated patient-actor conversations and nurse-in-the-loop data generation to train and align the system for bedside manner and medical reasoning.

Experimental results

Research questions

• RQ1 Can a safety-focused LLM constellation achieve parity with human nurses in medical safety, clinical readiness, patient education, conversational quality, and bedside manner during real-time conversations?
• RQ2 Do specialist support agents outperform a larger general-purpose LLM on healthcare-specific tasks and simulations?
• RQ3 How does task-specific modularization affect safety, latency, and accuracy in real-time nurse-assisted conversations?
• RQ4 What training and data strategies best align a primary conversational agent with nurse-like empathy and medical reasoning?
• RQ5 What are the operational challenges and safety trade-offs of multi-agent coordination in clinical dialogue?

Key findings

• The Polaris system achieves parity with human nurses on aggregate measures of medical safety, clinical readiness, patient education, conversational quality, and bedside manner in end-to-end evaluations.
• Specialist support agents significantly outperform a larger general-purpose LLM (GPT-4) and an LLM of similar size (LLaMA-2 70B) on healthcare tasks in component evaluations.
• The constellation architecture provides safety benefits through redundancy, specialization, and modular upgrades enabling safer, maintainable updates without retraining the entire system.
• Active safety guardrails with human supervision can be invoked when necessary, enhancing safety in high-stakes conversations.
• The primary agent focuses on conversational fluency and empathy, while specialist agents validate information, manage domain-specific tasks, and maintain state over long dialogues.