[Paper Review] Improving Medical Short Text Classification with Semantic Expansion Using Word-Cluster Embedding

Ying Shen, Qiang Zhang+4 more|arXiv (Cornell University)|Dec 5, 2018

Topic Modeling11 references4 citations

TL;DR

This paper proposes a semantic expansion method for medical short text classification by integrating word-cluster embeddings into deep neural networks. By clustering word vectors and using cluster centers to enrich semantic representations, the approach improves performance on both medical and general-domain short texts, outperforming state-of-the-art baselines on TREC and a newly released medical dataset.

ABSTRACT

Automatic text classification (TC) research can be used for real-world problems such as the classification of in-patient discharge summaries and medical text reports, which is beneficial to make medical documents more understandable to doctors. However, in electronic medical records (EMR), the texts containing sentences are shorter than that in general domain, which leads to the lack of semantic features and the ambiguity of semantic. To tackle this challenge, we propose to add word-cluster embedding to deep neural network for improving short text classification. Concretely, we first use hierarchical agglomerative clustering to cluster the word vectors in the semantic space. Then we calculate the cluster center vector which represents the implicit topic information of words in the cluster. Finally, we expand word vector with cluster center vector, and implement classifiers using CNN and LSTM respectively. To evaluate the performance of our proposed method, we conduct experiments on public data sets TREC and the medical short sentences data sets which is constructed and released by us. The experimental results demonstrate that our proposed method outperforms state-of-the-art baselines in short sentence classification on both medical domain and general domain.

Motivation & Objective

• To address the challenge of limited semantic features and ambiguity in short medical texts from electronic medical records (EMR).
• To improve the performance of deep neural network classifiers on short medical text classification tasks.
• To develop a semantic expansion technique that enhances word representations using cluster-level topic information.
• To evaluate the method on both general-domain (TREC) and domain-specific (medical) short text datasets.

Proposed method

• Apply hierarchical agglomerative clustering to word vectors in the semantic space to group semantically similar words.
• Compute the centroid (cluster center) of each word cluster to represent the implicit topic of the cluster.
• Expand individual word vectors by combining them with their corresponding cluster center vectors to enrich semantic features.
• Integrate the expanded word vectors into CNN and LSTM architectures for text classification.
• Train and fine-tune the classifiers end-to-end using the expanded representations.
• Use a newly constructed medical short sentence dataset alongside the public TREC dataset for evaluation.

Experimental results

Research questions

• RQ1 Can semantic expansion via word-cluster embedding improve the representation of short medical texts?
• RQ2 Does the integration of cluster-level topic information enhance classification performance on short texts?
• RQ3 How does the proposed method compare to state-of-the-art approaches on both medical and general-domain short text classification?
• RQ4 To what extent does the method mitigate semantic ambiguity and feature sparsity in short medical texts?

Key findings

• The proposed method achieves superior performance on the TREC dataset compared to state-of-the-art baselines in short text classification.
• On the newly released medical short sentence dataset, the method significantly outperforms existing approaches in terms of classification accuracy.
• The integration of cluster center vectors improves semantic richness, reducing ambiguity in short medical texts.
• Both CNN and LSTM models benefit from the semantic expansion, demonstrating the method's generalizability across architectures.