[Paper Review] Retrieving Evidence from EHRs with LLMs: Possibilities and Challenges

Hiba Ahsan, Denis Jered McInerney+5 more|PubMed|Sep 8, 2023

Machine Learning in HealthcareComputer Science31 references17 citations

TL;DR

The paper investigates using Flan-T5 XXL to retrieve and summarize unstructured EHR evidence for radiology-related diagnoses in a zero-shot setting, demonstrating that LLM outputs are preferred over a standard IR baseline but prone to hallucinations; confidence signals may help identify and mitigate hallucinations.

ABSTRACT

Unstructured data in Electronic Health Records (EHRs) often contains critical information-complementary to imaging-that could inform radiologists' diagnoses. But the large volume of notes often associated with patients together with time constraints renders manually identifying relevant evidence practically infeasible. In this work we propose and evaluate a zero-shot strategy for using LLMs as a mechanism to efficiently retrieve and summarize unstructured evidence in patient EHR relevant to a given query. Our method entails tasking an LLM to infer whether a patient has, or is at risk of, a particular condition on the basis of associated notes; if so, we ask the model to summarize the supporting evidence. Under expert evaluation, we find that this LLM-based approach provides outputs consistently preferred to a pre-LLM information retrieval baseline. Manual evaluation is expensive, so we also propose and validate a method using an LLM to evaluate (other) LLM outputs for this task, allowing us to scale up evaluation. Our findings indicate the promise of LLMs as interfaces to EHR, but also highlight the outstanding challenge posed by "hallucinations". In this setting, however, we show that model confidence in outputs strongly correlates with faithful summaries, offering a practical means to limit confabulations.

Motivation & Objective

• Motivate using LLMs to interface with unstructured EHR notes to aid radiologists in diagnosis.
• Evaluate a zero-shot prompting strategy to identify whether a patient has or is at risk of a condition and to summarize supporting evidence.
• Compare LLM-based retrieval to a neural-embedding baseline and assess expert (radiologist) judged usefulness and faithfulness of evidence.
• Characterize hallucinations in LLM outputs and explore model confidence signals as potential detectors.
• Discuss limitations and directions for future research in deploying EHR-interfacing LLMs safely.

Proposed method

• Use Flan-T5 XXL as the base LLM to perform zero-shot inference on clinical notes for specific diagnoses.
• Apply a sequential prompting strategy: first determine risk or presence of a diagnosis, then extract supporting evidence if positive.
• Compare against a retrieval baseline CBERT (neural embeddings) using GPT-3.5 to generate risk-factor phrases and ClinicalBERT for sentence embeddings.
• Evaluate evidence via expert radiologists who ground truth relevance and presence in notes, using a 0–3 usefulness scale.
• Analyze model confidence via LM likelihood and self-consistency prompts to detect hallucinations and correlate with usefulness.

Figure 1: Proposed prompting strategy to identify and summarize evidence relevant to a given query diagnosis using LLMs. We first ask if the patient has (or is at risk of) a condition, then elicit a summary of supporting evidence if so.

Experimental results

Research questions

• RQ1 Can a zero-shot LLM determine whether a patient is at risk of or has a given diagnosis from unstructured EHR notes?
• RQ2 What is the quality and faithfulness of LLM-generated evidence compared with a neural-embedding retrieval baseline?
• RQ3 How often do LLMs hallucinate evidence, and can confidence signals identify hallucinations effectively?
• RQ4 Do radiologists prefer abstractive LLM outputs over extractive baselines for surfacing and summarizing evidence?
• RQ5 How robust is the approach to variations in query terminology and dataset source?

Key findings

• LLM-generated evidence is often preferred by radiologists over the CBERT baseline for usefulness and conciseness.
• HALA: About 9.4% of FLAN-T5 evidence is hallucinated in the evaluated samples.
• Radiologists judged 41.5% (MIMIC) and 48.4% (BWH) of FLAN-T5 evidence as (very) useful, compared with 23.0% (MIMIC) and 18.5% (BWH) hallucinations.
• Model confidence scores (likelihood and self-consistency) robustly discriminate hallucinations (AUC > 0.9) and correlate with usefulness.
• Recall for identifying future diagnoses from notes achieved 0.7 in a subset study (140/200 correct).
• Radiologists reported that abstractive FLAN-T5 outputs provided more precise and concise summaries than extractive snippets.

Figure 3: Screenshot of the evaluation interface showing highlighted evidence.