[Paper Review] Large Language Models in the Clinic: A Comprehensive Benchmark

Andrew Liu, Li, Zheng+17 more|arXiv (Cornell University)|Apr 25, 2024

Artificial Intelligence in Healthcare6 citations

TL;DR

This paper introduces BenchHealth, a healthcare benchmark that evaluates 16 LLMs across medical reasoning, generation, and understanding with five trust-oriented metrics and human evaluation.

ABSTRACT

The adoption of large language models (LLMs) to assist clinicians has attracted remarkable attention. Existing works mainly adopt the close-ended question-answering (QA) task with answer options for evaluation. However, many clinical decisions involve answering open-ended questions without pre-set options. To better understand LLMs in the clinic, we construct a benchmark ClinicBench. We first collect eleven existing datasets covering diverse clinical language generation, understanding, and reasoning tasks. Furthermore, we construct six novel datasets and clinical tasks that are complex but common in real-world practice, e.g., open-ended decision-making, long document processing, and emerging drug analysis. We conduct an extensive evaluation of twenty-two LLMs under both zero-shot and few-shot settings. Finally, we invite medical experts to evaluate the clinical usefulness of LLMs. The benchmark data is available at https://github.com/AI-in-Health/ClinicBench.

Motivation & Objective

• Motivate trustworthy deployment of LLMs in clinical settings by moving beyond close-ended QA to open-ended medical tasks.
• Benchmark LLMs on diverse healthcare tasks using public datasets across reasoning, generation, and understanding.
• Introduce additional reliability-oriented metrics (faithfulness, comprehensiveness, robustness, generalizability) alongside traditional matching metrics.
• Provide a comparative analysis of general vs. medical LLMs and open-source vs. commercial models.
• Involve medical experts for human evaluation to assess clinical usefulness and limitations.

Proposed method

• Construct BenchHealth with seven tasks and thirteen public datasets spanning three scenarios: medical language reasoning, generation, and understanding.
• Evaluate sixteen LLMs (nine general, seven medical) under zero-shot and few-shot (1/3/5-shot) settings.
• Use five metrics beyond accuracy: faithfulness, comprehensiveness, robustness, generalizability, and traditional matching scores.
• Incorporate tailored prompts per task based on state-of-the-art prompts to optimize task understanding.
• Conduct human evaluation with medical experts comparing public vs. leading commercial LLMs on patient conversations.

Figure 1: Performance (measured by traditional matching scores) of LLMs under few (1,3,5)-shot learning setting.

Experimental results

Research questions

• RQ1 How do open-ended clinical tasks (reasoning, generation, understanding) differ in LLM performance from close-ended QA?
• RQ2 Do commercial LLMs outperform open-source models across healthcare tasks, and how do medical-fine-tuned LLMs compare to general LLMs?
• RQ3 How do model size, fine-tuning data, and few-shot learning affect performance, reliability, and clinical usefulness?
• RQ4 Which metrics best capture faithfulness, comprehensiveness, robustness, and generalizability in clinical contexts?
• RQ5 What is the relative clinical usefulness of general vs. medical LLMs according to human evaluation?

Key findings

• Commercial LLMs (e.g., GPT-4) outperform open-source models across tasks and datasets.
• All LLMs excel at close-ended QA but struggle on open-ended clinical decisions and some generation/understanding tasks.
• Fine-tuning general LLMs on medical data improves reasoning/understanding but can reduce summarization ability.
• Larger parameter counts generally improve performance across tasks; few-shot boosts reasoning and generation but may hurt understanding.
• Medical LLMs provide more faithful answers and better generalizability, while general LLMs offer broader comprehensiveness and robustness.
• Human evaluation shows medical LLMs surpass general LLMs in faithfulness and generalizability but are weaker in comprehensiveness and robustness compared to general LLMs.