[Paper Review] AgentClinic: a multimodal agent benchmark to evaluate AI in simulated clinical environments

Samuel Schmidgall, Rojin Ziaei+4 more|arXiv (Cornell University)|May 13, 2024

Impact of AI and Big Data on Business and Society18 citations

TL;DR

AgentClinic offers open-source multimodal and dialogue-based benchmarks to evaluate AI agents in simulated clinical settings, analyzing diagnostic accuracy, biases, and human-judged realism.

ABSTRACT

Evaluating large language models (LLM) in clinical scenarios is crucial to assessing their potential clinical utility. Existing benchmarks rely heavily on static question-answering, which does not accurately depict the complex, sequential nature of clinical decision-making. Here, we introduce AgentClinic, a multimodal agent benchmark for evaluating LLMs in simulated clinical environments that include patient interactions, multimodal data collection under incomplete information, and the usage of various tools, resulting in an in-depth evaluation across nine medical specialties and seven languages. We find that solving MedQA problems in the sequential decision-making format of AgentClinic is considerably more challenging, resulting in diagnostic accuracies that can drop to below a tenth of the original accuracy. Overall, we observe that agents sourced from Claude-3.5 outperform other LLM backbones in most settings. Nevertheless, we see stark differences in the LLMs' ability to make use of tools, such as experiential learning, adaptive retrieval, and reflection cycles. Strikingly, Llama-3 shows up to 92% relative improvements with the notebook tool that allows for writing and editing notes that persist across cases. To further scrutinize our clinical simulations, we leverage real-world electronic health records, perform a clinical reader study, perturb agents with biases, and explore novel patient-centric metrics that this interactive environment firstly enables.

Motivation & Objective

• Motivate evaluation of AI in sequential, interactive clinical decision making beyond static QA benchmarks.
• Provide open benchmarks (AgentClinic-MedQA and AgentClinic-NEJM) that use patient, doctor, measurement, and moderator agents.
• Embed cognitive and implicit biases in agents to study their impact on diagnostics and patient perceptions.
• Assess how different LLMs perform as doctor agents and how patient language models influence outcomes.
• Offer human clinician annotations to rate realism and empathy in agent interactions.

Proposed method

• Create four language agents (patient, doctor, measurement, moderator) with role-specific prompts and access to structured scenario data.
• Use USMLE-derived questions (MedQA) and NEJM cases to build OSCE-like, structured JSON scenarios for doctor-patient dialogues.
• Introduce bias prompts (cognitive and implicit) for doctor and patient agents and measure impacts on accuracy and patient perceptions.
• Evaluate both monomodal dialogue (AgentClinic-MedQA) and multimodal image-plus-dialogue (AgentClinic-NEJM) settings with different LLMs.
• Conduct bias analyses comparing GPT-4 and Mixtral-8x7B across accuracy and perception metrics.
• Incorporate a moderator to parse unstructured doctor outputs and determine diagnostic correctness.

Figure 1: Composing and running language agents in AgentClinic. (Left) Agents are composed of several elements in AgentClinic: an LLM backbone, context, a role, and potential biases. Each of these different elements can be modified to create an unlimited number of unique language agents that can act

Experimental results

Research questions

• RQ1 How well do state-of-the-art LLMs diagnose from physician-patient dialogues in AgentClinic-MedQA?
• RQ2 How do cognitive and implicit biases in doctor and patient agents affect diagnostic accuracy and patient perceptions?
• RQ3 Does the patient agent's underlying language model influence diagnostic success and interaction dynamics?
• RQ4 What is the impact of limited versus excessive interaction turns on doctor diagnostic accuracy in AgentClinic?
• RQ5 How does multimodal image data (AgentClinic-NEJM) affect diagnosis compared to text-only dialogue?

Key findings

• GPT-4 achieves 52% accuracy on AgentClinic-MedQA; GPT-4o 38%; Mixtral-8x7B 37%; Llama 3 70B-instruct 30%; Llama 2 70B-chat 9%.
• On AgentClinic-NEJM multimodal tasks, GPT-4o achieves 47% accuracy when images are provided initially, while GPT-4-turbo and GPT-4-vision-preview reach 27% each; providing images upon request reduces accuracy further (GPT-4o 27%, GPT-4-turbo 20%, GPT-4-vision-preview 13%).
• Bias introduction reduces doctor and patient diagnostic accuracy and lowers patient compliance, confidence, and willingness for follow-up consultations; GPT-4 shows smaller accuracy drops compared to Mixtral-8x7B.
• The patient agent's language model critically influences diagnostic success; cross-model interactions (doctor vs patient) can introduce additional challenges.
• Limited (N=10–15) or excessive (N=25–30) interaction turns both reduce diagnostic accuracy compared to a balanced 20-turn setting.
• Human clinician evaluators rated realism and empathy moderately (doctor 6.2/10, patient 6.7/10, measurement 6.3/10, empathy 5.8/10).
• AgentClinic-MedQA accuracy is not strongly predicted by MedQA scores alone, highlighting the complexity of dialogue-based clinical reasoning.

Figure 2: Process of conversion from USMLE question, to AgentClinic-MedQA Objective Structured Clinical Examination (OSCE) template, to building a patient agent that is powered by a large language model (LLM).