[Paper Review] Towards Automated ICD Coding Using Deep Learning

Haoran Shi, Pengtao Xie+3 more|arXiv (Cornell University)|Nov 11, 2017

Biomedical Text Mining and OntologiesBiochemistry, Genetics and Molecular Biology26 references127 citations

TL;DR

The paper presents a hierarchical deep learning model with attention to automatically assign ICD codes from diagnosis descriptions, achieving a Soft-attention F1 of 0.532 and AUC-ROC of 0.900 on MIMIC-III data.

ABSTRACT

International Classification of Diseases(ICD) is an authoritative health care classification system of different diseases and conditions for clinical and management purposes. Considering the complicated and dedicated process to assign correct codes to each patient admission based on overall diagnosis, we propose a hierarchical deep learning model with attention mechanism which can automatically assign ICD diagnostic codes given written diagnosis. We utilize character-aware neural language models to generate hidden representations of written diagnosis descriptions and ICD codes, and design an attention mechanism to address the mismatch between the numbers of descriptions and corresponding codes. Our experimental results show the strong potential of automated ICD coding from diagnosis descriptions. Our best model achieves 0.53 and 0.90 of F1 score and area under curve of receiver operating characteristic respectively. The result outperforms those achieved using character-unaware encoding method or without attention mechanism. It indicates that our proposed deep learning model can code automatically in a reasonable way and provide a framework for computer-auxiliary ICD coding.

Motivation & Objective

• Motivate automatic ICD coding to reduce coding errors and costs in healthcare.
• Formulate ICD coding as a multi-label classification problem on diagnosis descriptions.
• Develop a neural architecture that bridges the stylistic gap between diagnosis texts and ICD code definitions.
• Evaluate whether attention mechanisms improve alignment between diagnosis descriptions and ICD codes.

Proposed method

• Encode diagnosis descriptions using character-level and word-level LSTM networks to obtain hidden representations.
• Encode ICD code definitions (long titles) with a parallel character- and word-level LSTM to obtain code representations.
• Compute attention scores between each ICD code and diagnosis descriptions using cosine similarity of hidden states.
• Apply soft-attention to aggregate diagnosis descriptions into a code-specific vector, then project to a probability via a sigmoid output layer.
• Train with binary cross-entropy loss using Adam optimizer, tuning thresholds for best F1 on validation data.

Experimental results

Research questions

• RQ1 Can a hierarchical neural model with attention effectively map free-text diagnosis descriptions to multiple ICD codes?
• RQ2 Does soft-attention outperform hard-selection in aligning diagnosis descriptions with ICD code definitions?
• RQ3 How do character-level encoders contribute to robust representations of medical terms and typos?
• RQ4 What is the impact of using ICD-9 code definitions for coding performance on a dataset like MIMIC-III?

Key findings

• Soft-attention improves F1 to 0.532 and AUC-ROC to 0.900, outperforming the hard-selection model.
• Hard-selection yields F1 of 0.480 and AUC-ROC of 0.877.
• Ablation studies show that character-level encoding and attention are both crucial for performance.
• Replacing character-level LSTM with random or non-character encoders degrades F1 and AUC-ROC.
• Using pre-trained word embeddings helps but does not surpass the character-level encoder in this setup.
• Attention visualization demonstrates varying focus across diagnosis descriptions for different ICD codes.