[Paper Review] Evaluating large language models in medical applications: a survey

Xiaolan Chen, Jiayang Xiang+4 more|arXiv (Cornell University)|May 13, 2024

Artificial Intelligence in Healthcare and Education4 citations

TL;DR

This survey evaluates large language models (LLMs) in medical applications by synthesizing existing research on evaluation data sources, task scenarios, and methodologies. It identifies critical challenges in medical LLM evaluation and calls for standardized, reliable assessment frameworks to ensure clinical safety and effectiveness.

ABSTRACT

Large language models (LLMs) have emerged as powerful tools with transformative potential across numerous domains, including healthcare and medicine. In the medical domain, LLMs hold promise for tasks ranging from clinical decision support to patient education. However, evaluating the performance of LLMs in medical contexts presents unique challenges due to the complex and critical nature of medical information. This paper provides a comprehensive overview of the landscape of medical LLM evaluation, synthesizing insights from existing studies and highlighting evaluation data sources, task scenarios, and evaluation methods. Additionally, it identifies key challenges and opportunities in medical LLM evaluation, emphasizing the need for continued research and innovation to ensure the responsible integration of LLMs into clinical practice.

Motivation & Objective

• To provide a comprehensive synthesis of current approaches to evaluating large language models (LLMs) in medical contexts.
• To identify and categorize key evaluation data sources used in medical LLM research.
• To analyze prevalent task scenarios such as clinical decision support, diagnosis, and patient education.
• To examine existing evaluation methods, including automatic metrics and human annotation.
• To highlight critical challenges and future research needs for responsible deployment of LLMs in clinical settings.

Proposed method

• Systematic review and synthesis of existing studies on medical LLM evaluation from the literature.
• Categorization of evaluation data sources into clinical datasets, synthetic data, and public benchmarks.
• Classification of task scenarios into clinical reasoning, documentation, patient interaction, and knowledge grounding.
• Analysis of evaluation methods, including automated metrics (e.g., BLEU, ROUGE), factuality scoring, and human evaluation protocols.
• Identification of gaps in current evaluation practices through comparative analysis of methodological approaches.
• Synthesis of challenges such as data quality, hallucination, and clinical safety, with recommendations for future research.

Experimental results

Research questions

• RQ1 What are the primary data sources used to evaluate LLMs in medical applications?
• RQ2 Which clinical tasks are most commonly evaluated in current medical LLM research?
• RQ3 What evaluation methods are predominantly used, and how do they compare in reliability and validity?
• RQ4 What are the major challenges in evaluating LLMs for medical use, and how do they impact clinical trustworthiness?
• RQ5 What opportunities exist for improving evaluation frameworks to support safe and effective deployment in healthcare?

Key findings

• A wide diversity of data sources is used in medical LLM evaluation, including real clinical records, synthetic data, and curated benchmarks.
• Common evaluation tasks include clinical question answering, medical coding, and patient response generation.
• Automatic metrics like BLEU and ROUGE show limited correlation with clinical relevance, highlighting the need for better evaluation criteria.
• Human evaluation remains a gold standard but is resource-intensive and inconsistent across studies.
• Hallucination and factual inaccuracy are persistent challenges, especially in complex diagnostic reasoning tasks.
• There is a lack of standardized evaluation protocols, creating barriers to reproducibility and clinical adoption.