[Paper Review] Query-Focused EHR Summarization to Aid Imaging Diagnosis

Denis Jered McInerney, Borna E. Dabiri+4 more|arXiv (Cornell University)|Apr 9, 2020

Topic Modeling23 references4 citations

TL;DR

This paper proposes a distantly supervised, query-focused extractive summarization model for electronic health records (EHRs) that aids radiologists in diagnosing medical imaging by retrieving clinically relevant text snippets conditioned on potential diagnoses (e.g., ICD codes). Using Clinical BERT and a transformer-based attention mechanism, the model leverages future ICD codes as noisy supervision to identify salient EHR content, significantly outperforming unsupervised baselines in radiologist evaluations.

ABSTRACT

Electronic Health Records (EHRs) provide vital contextual information to radiologists and other physicians when making a diagnosis. Unfortunately, because a given patient's record may contain hundreds of notes and reports, identifying relevant information within these in the short time typically allotted to a case is very difficult. We propose and evaluate models that extract relevant text snippets from patient records to provide a rough case summary intended to aid physicians considering one or more diagnoses. This is hard because direct supervision (i.e., physician annotations of snippets relevant to specific diagnoses in medical records) is prohibitively expensive to collect at scale. We propose a distantly supervised strategy in which we use groups of International Classification of Diseases (ICD) codes observed in 'future' records as noisy proxies for 'downstream' diagnoses. Using this we train a transformer-based neural model to perform extractive summarization conditioned on potential diagnoses. This model defines an attention mechanism that is conditioned on potential diagnoses (queries) provided by the diagnosing physician. We train (via distant supervision) and evaluate variants of this model on EHR data from Brigham and Women's Hospital in Boston and MIMIC-III (the latter to facilitate reproducibility). Evaluations performed by radiologists demonstrate that these distantly supervised models yield better extractive summaries than do unsupervised approaches. Such models may aid diagnosis by identifying sentences in past patient reports that are clinically relevant to a potential diagnosis.

Motivation & Objective

• To address the challenge of radiologists lacking time to review voluminous EHR notes during imaging interpretation.
• To develop a method for generating extractive, query-focused summaries of EHR notes without requiring expensive direct annotation of snippet relevance.
• To leverage future ICD codes as distant supervision to train models that identify clinically relevant text snippets for potential diagnoses.
• To evaluate whether distantly supervised models can outperform unsupervised summarization approaches in clinical relevance and utility.
• To enable efficient, scalable access to relevant patient history during point-of-care diagnosis, particularly in radiology.

Proposed method

• The model uses Clinical BERT to jointly encode a diagnostic query (e.g., ICD code description) and EHR notes for contextual representation.
• A transformer-based architecture with a pointer mechanism generates extractive summaries by attending over input sentences based on the query.
• Distant supervision is applied by using groups of future ICD codes as proxy labels for downstream diagnoses to train the model.
• The model is trained to predict future ICD code groups, using the top attention weights as a soft scoring mechanism to retrieve relevant text snippets.
• The attention mechanism is conditioned on the query (diagnosis) and produces a distribution over input sentences to identify the most relevant ones.
• An interface was developed to allow radiologists to annotate snippet relevance and evaluate model outputs in a blinded manner.

Experimental results

Research questions

• RQ1 Can distantly supervised models generate more clinically relevant EHR summaries than unsupervised methods for radiology diagnosis?
• RQ2 To what extent can future ICD codes serve as effective noisy proxies for training query-focused summarization models?
• RQ3 How do radiologists perceive the relevance and utility of model-generated summaries for diagnostic reasoning?
• RQ4 Can a model conditioned on a diagnostic query retrieve salient EHR content without direct supervision on snippet relevance?
• RQ5 Does the use of a transformer-based attention mechanism conditioned on diagnostic queries improve extractive summarization performance?

Key findings

• Radiologists rated the distantly supervised models as significantly more relevant and useful than unsupervised baselines in identifying clinically relevant EHR snippets for diagnostic queries.
• The model’s attention mechanism effectively highlighted sentences from past notes that were predictive of future diagnoses, even without explicit supervision on snippet relevance.
• The use of future ICD codes as distant supervision enabled effective training of a query-focused summarization model with no need for manual annotation of relevant snippets.
• The proposed model outperformed unsupervised methods in both automated evaluation and radiologist assessment, demonstrating improved clinical utility.
• The system demonstrated feasibility for real-time, point-of-care support in radiology by retrieving contextually relevant patient history from EHRs.
• The approach is generalizable beyond radiology, as it relies only on existing ICD codes and does not require task-specific annotation.