[Paper Review] Multi-Label Learning from Medical Plain Text with Convolutional Residual Models

Xinyuan Zhang, Ricardo Henao+3 more|arXiv (Cornell University)|Jan 15, 2018

Text and Document Classification TechnologiesComputer Science19 citations

TL;DR

This paper proposes a convolutional residual network that combines a CNN encoder for medical text representation with a deep residual classifier to model label dependencies in multi-label diagnosis prediction from clinical notes. The model outperforms baselines across multiple metrics, achieving state-of-the-art performance on an EHR dataset by effectively capturing semantic structure and comorbidity patterns in long, noisy doctor notes.

ABSTRACT

Predicting diagnoses from Electronic Health Records (EHRs) is an important medical application of multi-label learning. We propose a convolutional residual model for multi-label classification from doctor notes in EHR data. A given patient may have multiple diagnoses, and therefore multi-label learning is required. We employ a Convolutional Neural Network (CNN) to encode plain text into a fixed-length sentence embedding vector. Since diagnoses are typically correlated, a deep residual network is employed on top of the CNN encoder, to capture label (diagnosis) dependencies and incorporate information directly from the encoded sentence vector. A real EHR dataset is considered, and we compare the proposed model with several well-known baselines, to predict diagnoses based on doctor notes. Experimental results demonstrate the superiority of the proposed convolutional residual model.

Motivation & Objective

• To address the challenge of multi-label diagnosis prediction from unstructured doctor notes in Electronic Health Records (EHRs), where patients often have multiple comorbidities.
• To improve performance by modeling label dependencies, as traditional methods assume label independence and fail to capture comorbidity patterns.
• To develop a deep learning architecture that effectively encodes long, noisy clinical text while preserving semantic and syntactic structure.
• To demonstrate that residual networks enhance multi-label classification by integrating sentence-level representations with learned label correlations.
• To validate the model on a real-world EHR dataset and compare it against multiple strong baselines, including CNN, Bi-LSTM, and tree-based methods.

Proposed method

• A Convolutional Neural Network (CNN) is used to encode plain-text doctor notes into fixed-length sentence embeddings, capturing local semantics and word order.
• A deep residual network is stacked on top of the CNN encoder, using shortcut connections to allow training of deeper architectures and model label dependencies.
• The residual network integrates information from the encoded sentence vector and learns to predict multiple diagnoses simultaneously by capturing co-occurrence patterns among labels.
• The model is trained end-to-end using cross-entropy loss with sigmoid activation for multi-label classification, enabling independent prediction of each diagnosis.
• t-SNE visualization is used to evaluate the quality of learned sentence embeddings, showing clusters of semantically similar diagnoses.
• The architecture is evaluated across varying residual layer depths (1 to 8) to study the impact of depth on performance and convergence speed.

Experimental results

Research questions

• RQ1 Can a CNN-based sentence encoder outperform traditional bag-of-words and LSTM-based representations in capturing semantic meaning from long, noisy clinical notes?
• RQ2 Does incorporating label dependencies through a deep residual network improve multi-label diagnosis prediction accuracy compared to independent label prediction?
• RQ3 How does the depth of the residual network affect performance, convergence speed, and generalization in multi-label EHR classification?
• RQ4 To what extent does the proposed model leverage comorbidity patterns, particularly for diagnoses that commonly co-occur?
• RQ5 Is the performance gain due to increased model capacity or due to the architectural innovation of residual connections?

Key findings

• The 8-layer convolutional residual model achieved the highest performance across all metrics, with a P@1 of 60.30%, P@3 of 31.21%, P@5 of 21.27%, nDCG@3 of 80.21%, nDCG@5 of 80.17%, and AUC of 94.89% on the ehr1000 validation set.
• The model with 8 residual layers outperformed the plain 8-layer CNN, which achieved only 8.99% P@1, indicating that residual connections are key to performance, not just depth.
• The residual model converged faster than shallower variants, requiring only 25 epochs compared to 38 for the 1-layer residual model.
• The AUC of the 8-layer residual model (0.942) significantly exceeded that of the standard CNN (0.923) on the ehr64-all dataset, with 59 out of 64 diagnoses showing improved AUC.
• t-SNE visualization confirmed that sentence embeddings cluster by diagnosis, with semantically similar conditions (e.g., headache and migraine) grouped together.
• The model showed the greatest performance gains for diagnoses with high comorbidity rates, such as hyperglycemia, hypotension, hyperkalemia, UTI, and fever, indicating effective modeling of label dependencies.