[Paper Review] Assigning Medical Codes at the Encounter Level by Paying Attention to Documents

Han-Chin Shing, Guoli Wang+1 more|arXiv (Cornell University)|Nov 15, 2019

Topic Modeling9 references4 citations

TL;DR

This paper proposes Encounter-Level Document Attention Networks (ELDAN), a hierarchical attention model that improves medical coding accuracy at the encounter level by identifying relevant documents within multi-document clinical encounters. It achieves state-of-the-art performance without requiring document-level annotations, significantly outperforming baselines and enabling interpretable code review by highlighting source documents.

ABSTRACT

The vast majority of research in computer assisted medical coding focuses on coding at the document level, but a substantial proportion of medical coding in the real world involves coding at the level of clinical encounters, each of which is typically represented by a potentially large set of documents. We introduce encounter-level document attention networks, which use hierarchical attention to explicitly take the hierarchical structure of encounter documentation into account. Experimental evaluation demonstrates improvements in coding accuracy as well as facilitation of human reviewers in their ability to identify which documents within an encounter play a role in determining the encounter level codes.

Motivation & Objective

• To address the gap in computer-assisted medical coding (CAC) by focusing on encounter-level coding rather than document-level coding.
• To improve coding accuracy by modeling the hierarchical structure of clinical encounters, which consist of multiple documents.
• To enhance interpretability by enabling human coders to identify which documents support each encounter-level code.
• To develop a method that does not require expensive document-level annotations, leveraging only encounter-level codes during training.
• To evaluate the model’s ability to assign attention weights that align with human judgments of source documents.

Proposed method

• ELDAN uses a hierarchical attention mechanism that processes text at three levels: words within sentences, sentences within documents, and documents within clinical encounters.
• It transforms raw clinical text into feature vectors using UMLS CUIs and internal concept identifiers, preserving patient privacy.
• The model applies multi-level attention to learn which documents are most relevant for predicting encounter-level codes.
• It uses a one-vs-all classification setup with a shared encoder for all codes, enabling efficient training and transfer learning for rare codes.
• Document-level attention scores are used to predict source documents, and these are evaluated against human annotations.
• The model is trained end-to-end on encounter-level codes, avoiding the need for document-level code annotations.

Experimental results

Research questions

• RQ1 Can a deep learning model that leverages the hierarchical structure of clinical encounters improve coding accuracy at the encounter level?
• RQ2 Does the model’s document-level attention mechanism align with human judgments of source documents for encounter codes?
• RQ3 Can the model achieve high performance without requiring document-level annotations?
• RQ4 How effective is a naïve transfer learning approach for rare medical codes in this setting?
• RQ5 Can the attention mechanism serve as a form of weak supervision to guide human review or active learning?

Key findings

• ELDAN significantly outperforms a strong baseline in encounter-level coding accuracy, with statistically significant improvements (p < .05).
• For the 20 most frequent CPT codes, ELDAN achieved a 100% F1-score on 7 out of 19 codes, demonstrating strong performance on high-frequency codes.
• On average, ELDAN’s document attention F1-score exceeded the chance baseline by 15.5 points, with a 36.29-point improvement for code 12002.
• The model achieved a 42.38-point F1-score improvement over chance for code 43235, despite only 6 encounters and 18 documents.
• The attention mechanism showed strong alignment with human coders’ source-document judgments, with a 29.67-point F1-score difference for code 43239.
• The model demonstrated effective transfer learning, with a naïve approach significantly improving performance on rare codes without fine-tuning.