[Paper Review] Can large language models reason about medical questions?

Valentin Liévin, Christoffer Hother+2 more|arXiv (Cornell University)|Jul 17, 2022

Topic Modeling85 citations

TL;DR

The paper evaluates GPT-3.5 family and open-source LLMs on medical QA benchmarks using zero-/few-shot and Chain-of-Thought prompting, with and without retrieval augmentation, showing near-passing performance on USMLE, MedMCQA, and PubMedQA under certain prompting/compute settings.

ABSTRACT

Although large language models (LLMs) often produce impressive outputs, it remains unclear how they perform in real-world scenarios requiring strong reasoning skills and expert domain knowledge. We set out to investigate whether close- and open-source models (GPT-3.5, LLama-2, etc.) can be applied to answer and reason about difficult real-world-based questions. We focus on three popular medical benchmarks (MedQA-USMLE, MedMCQA, and PubMedQA) and multiple prompting scenarios: Chain-of-Thought (CoT, think step-by-step), few-shot and retrieval augmentation. Based on an expert annotation of the generated CoTs, we found that InstructGPT can often read, reason and recall expert knowledge. Last, by leveraging advances in prompt engineering (few-shot and ensemble methods), we demonstrated that GPT-3.5 not only yields calibrated predictive distributions, but also reaches the passing score on three datasets: MedQA-USMLE 60.2%, MedMCQA 62.7% and PubMedQA 78.2%. Open-source models are closing the gap: Llama-2 70B also passed the MedQA-USMLE with 62.5% accuracy.

Motivation & Objective

• Assess how GPT-3.5-series models (InstructGPT, Codex) perform on medical QA benchmarks (USMLE, MedMCQA, PubMedQA).
• Evaluate zero-shot, few-shot, direct, and Chain-of-Thought prompting, plus retrieval augmentation, for medical reasoning.
• Evaluate open-source models (Llama-2, Vicuna, Guanaco, Falcon, etc.) on the same benchmarks and analyze calibration and biases.

Proposed method

• Compare zero-shot, direct, and zero-shot CoT prompting on Medical QA datasets.
• Incorporate few-shot prompts with and without CoT explanations.
• Experiment with retrieval-augmented prompting using a BM25 retriever and Wikipedia passages.
• Analyze ensemble sampling (self-consistency) and inference-time compute (Codex 5-shot CoT with k=100) for uncertainty and calibration.
• Perform expert annotation of generated CoTs to assess reasoning, knowledge recall, and reading comprehension.
• Benchmark open-source models (7B–70B) against Codex on the MedQA-USMLE and MedMCQA datasets.

Experimental results

Research questions

• RQ1 Can InstructGPT and Codex achieve passing accuracy on MedQA-USMLE, MedMCQA, and PubMedQA using zero-shot and few-shot prompting?
• RQ2 Does chain-of-thought prompting improve medical question answering accuracy and interpretability compared to direct prompts?
• RQ3 Can retrieval augmentation and ensemble sampling improve accuracy and calibration?
• RQ4 How do open-source LLMs compare to proprietary models on these medical benchmarks?
• RQ5 What biases and failure modes emerge in medical reasoning prompts and CoT outputs?

Key findings

• GPT-3.5 family with prompting can reach passing scores on three medical datasets under certain prompting and compute conditions.
• Codex 5-shot CoT with ensemble sampling (k=100) passes USMLE (60.2%), MedMCQA (62.7%), and PubMedQA (78.2%).
• Open-source models (e.g., Llama-2 70B) can approach or surpass some closed models in USMLE performance (e.g., 62.5% with 5-shot CoT) and narrow the gap overall.
• Zero-shot direct prompting outperforms finetuned BERT baselines on these medical benchmarks; zero-shot CoT prompts provide interpretable reasoning but not always higher accuracy than direct prompts.
• Retrieval augmentation provides incremental gains; ensemble/self-consistency sampling improves reliability and calibration.
• Human expert evaluation indicates InstructGPT can read, reason, and recall expert knowledge in many CoTs, though errors and bias remain.