[Paper Review] Seeing The Whole Patient: Using Multi-Label Medical Text Classification Techniques to Enhance Predictions of Medical Codes

Vithya Yogarajan, Jacob Montiel+2 more|arXiv (Cornell University)|Mar 29, 2020

Biomedical Text Mining and Ontologies40 references4 citations

TL;DR

This paper proposes using high-dimensional, health-specific word embeddings—particularly Skip-gram models with 600 dimensions—to improve multi-label medical text classification for predicting ICD-9 codes in patients with multi-morbidity. It demonstrates that such embeddings significantly boost F-measure, especially for rare medical codes, and shows minimal benefit from text pre-processing on de-identified EHR data.

ABSTRACT

Machine learning-based multi-label medical text classifications can be used to enhance the understanding of the human body and aid the need for patient care. We present a broad study on clinical natural language processing techniques to maximise a feature representing text when predicting medical codes on patients with multi-morbidity. We present results of multi-label medical text classification problems with 18, 50 and 155 labels. We compare several variations to embeddings, text tagging, and pre-processing. For imbalanced data we show that labels which occur infrequently, benefit the most from additional features incorporated in embeddings. We also show that high dimensional embeddings pre-trained using health-related data present a significant improvement in a multi-label setting, similarly to the way they improve performance for binary classification. High dimensional embeddings from this research are made available for public use.

Motivation & Objective

• To improve the prediction of multiple concurrent medical codes (ICD-9) in patients with multi-morbidity using multi-label text classification.
• To investigate how different embedding techniques—especially dimensionality and architecture (Skip-gram vs. CBOW)—affect performance on imbalanced medical text data.
• To evaluate the impact of pre-processing free-form clinical text on classification performance, especially in de-identified datasets like MIMIC-III.
• To assess the effectiveness of various tagging strategies (e.g., POS, text-split tags) and classifier configurations in multi-label medical NLP tasks.
• To provide publicly available, high-quality, pre-trained medical text embeddings for broader research use in clinical NLP.

Proposed method

• Trained word embeddings using Skip-gram and CBOW models on de-identified clinical notes from the MIMIC-III dataset, with varying embedding dimensions (300, 600) and pre-training strategies.
• Used a multi-label classification framework with logistic regression and elastic net regularization (ECC) as the base classifier for predicting ICD-9 codes.
• Explored multiple embedding variants: W300 (Word2Vec), T300/T600 (fastText with 300/600 dimensions), CONCAT300/350 (concatenated embeddings), and embeddings with POS or text-split tagging.
• Applied micro- and macro-averaged F1 scores to evaluate performance across 18, 50, and 155 ICD-9 code labels, with separate analysis by label frequency.
• Compared pre-processed text (tokenized, truncated to 2500 tokens) against raw text to assess pre-processing utility in de-identified EHRs.
• Conducted ablation studies on tagging methods and embedding types to isolate performance gains from specific design choices.

Experimental results

Research questions

• RQ1 How do different embedding architectures (e.g., Skip-gram vs. CBOW) affect multi-label medical text classification performance on rare and common ICD-9 codes?
• RQ2 To what extent does increasing embedding dimensionality (e.g., 300 vs. 600) improve F-measure, especially for infrequently occurring medical codes?
• RQ3 Does pre-processing free-form clinical text (e.g., tokenization, truncation) lead to measurable improvements in F-measure compared to using raw text?
• RQ4 How do different tagging strategies (POS, text-split tags) influence the performance of multi-label medical text classification models?
• RQ5 What is the impact of combining multiple embedding types (e.g., CONCAT300) on overall classification performance in multi-morbidity prediction tasks?

Key findings

• Skip-gram models with 600-dimensional embeddings (T600SG) achieved the highest micro- and macro-averaged F1 scores across all label sets (18, 50, 155), with a macro F1 of 0.674 on the 155-label task.
• For rare ICD-9 codes, such as 410.71 (F-measure from 0.332 to 0.404) and 410.70 (from 0.329 to 0.404), high-dimensional embeddings (T600SG) provided the largest relative improvement.
• Pre-processing of clinical text (e.g., truncation to 2500 tokens or standard tokenization) yielded only marginal gains in F-measure compared to using raw text, suggesting minimal benefit on already de-identified data.
• T300 embeddings outperformed W300 across all label sets, with macro F1 improvements of 0.005 (18 labels), 0.011 (50 labels), and 0.013 (155 labels), indicating that higher-dimensional embeddings are consistently better.
• The use of text-split tagging (T300, TextSplitTag) improved macro F1 to 0.684 on the 155-label task, outperforming standard POS tagging (0.646), suggesting that structured tagging can enhance representation.
• The best-performing model (T600SG) achieved a micro F1 of 0.745 and macro F1 of 0.674 on the 155-label task, demonstrating that high-dimensional, health-optimized embeddings significantly enhance multi-label prediction in multi-morbidity settings.