[Paper Review] Multi-Label Classification of Patient Notes a Case Study on ICD Code Assignment

Tal Baumel, Jumana Nassour-Kassis+3 more|arXiv (Cornell University)|Sep 27, 2017

Biomedical Text Mining and OntologiesBiochemistry, Genetics and Molecular Biology88 citations

TL;DR

This paper compares four models for assigning multiple ICD codes from discharge summaries and shows hierarchical attention GRU (HA-GRU) achieving state-of-the-art results with transparent, sentence-level attention for error analysis. It uses MIMIC II and III datasets with full and rolled-up ICD-9 codes.

ABSTRACT

In the context of the Electronic Health Record, automated diagnosis coding of patient notes is a useful task, but a challenging one due to the large number of codes and the length of patient notes. We investigate four models for assigning multiple ICD codes to discharge summaries taken from both MIMIC II and III. We present Hierarchical Attention-GRU (HA-GRU), a hierarchical approach to tag a document by identifying the sentences relevant for each label. HA-GRU achieves state-of-the art results. Furthermore, the learned sentence-level attention layer highlights the model decision process, allows easier error analysis, and suggests future directions for improvement.

Motivation & Objective

• Motivate automatic, multi-label ICD coding from long clinical notes in EHRs with a very large label set.
• Investigate four models (SVM, CBOW, CNN, HA-GRU) on full and rolled-up ICD-9 code sets.
• Provide model transparency through attention mechanisms to explain predictions.
• Evaluate models on MIMIC II and MIMIC III to assess training data size effects and generalizability.

Proposed method

• Evaluate four models for multi-label ICD coding: one-vs-all SVM, CBOW neural model, CNN, and HA-GRU with hierarchical sentence-level attention.
• Preprocess text with spaCy tokenization, edit-distance based normalization, and a 5+ token vocabulary cutoff.
• Segment documents hierarchically into sentences to enable a two-level GRU encoder.
• HA-GRU applies a sentence-level bidirectional GRU with attention to select relevant sentences per label, followed by a document-level bidirectional GRU with label-specific attention.
• Train each classifier separately using appropriate loss: binary cross-entropy for SVM/CBOW/CNN and categorical cross-entropy for HA-GRU, with softmax outputs per label.
• Compare models under two label configurations: full 5-digit ICD-9 codes and rolled-up 3-digit codes.

Experimental results

Research questions

• RQ1 Can extreme multi-label classification be effectively performed on long discharge summaries with a very large ICD code set?
• RQ2 Does hierarchical attention in a DRNN (HA-GRU) improve multi-label ICD code assignment over traditional baselines?
• RQ3 How do models perform on full ICD-9 vs rolled-up ICD-9 code sets and with different training data sizes (MIMIC II vs MIMIC III)?
• RQ4 Can attention mechanisms provide transparent, interpretable explanations for ICD code predictions?

Key findings

• HA-GRU achieves the best performance for rolled-up ICD-9 codes, with a 55.86% Micro-F1 on MIMIC III, surpassing the best non-HA baselines by about 2.8 percentage points.
• In full ICD-9 experiments, CNN slightly outperforms HA-GRU on MIMIC III, but HA-GRU remains strong and provides better explanation capabilities.
• When trained on MIMIC II vs MIMIC III, all models improve with more data, and HA-GRU shows substantial gains on the rolled-up task (e.g., 7% absolute F1 improvement on MIMIC II in rolled-up setting).
• Two-level attention enables per-label sentence selection and per-sentence word-level focus, aiding model transparency and error analysis.
• Preprocessing steps that tokenize and normalize rare variants via edit distance significantly improve CBOW and CNN performance by about 0.5% F1.