[Paper Review] Generating Continuous Representations of Medical Texts

Graham Spinks, Marie‐Francine Moens|Lirias (KU Leuven)|May 15, 2018

Topic Modeling6 references3 citations

TL;DR

This paper proposes a conditional Adversarially Regularized Autoencoder (ARAE) for generating realistic medical text captions from continuous representations, improving upon traditional LSTMs. The method learns disentangled, informative continuous embeddings while generating high-quality, syntactically accurate reports, achieving a 125.4 perplexity—significantly lower than the LSTM baseline (150.0) and the unconditional ARAE (148.4).

ABSTRACT

We present an architecture that generates medical texts while learning an informative, continuous representation with discriminative features. During training the input to the system is a dataset of captions for medical X-Rays. The acquired continuous representations are of particular interest for use in many machine learning techniques where the discrete and high-dimensional nature of textual input is an obstacle. We use an Adversarially Regularized Autoencoder to create realistic text in both an unconditional and conditional setting. We show that this technique is applicable to medical texts which often contain syntactic and domain-specific shorthands. A quantitative evaluation shows that we achieve a lower model perplexity than a traditional LSTM generator.

Motivation & Objective

• To develop a deep generative model that learns continuous, disentangled representations from medical text while generating realistic reports.
• To address the challenge of training GANs on discrete text data by adapting the ARAE framework to medical reports with domain-specific terminology and syntactic irregularities.
• To improve text generation quality over standard LSTMs by incorporating adversarial training and conditional supervision using MeSH labels.
• To demonstrate that continuous representations from ARAE can be used to generate diverse, medically plausible captions while maintaining semantic coherence.
• To evaluate the model’s performance quantitatively using perplexity and qualitatively through a user-facing demo interface.

Proposed method

• The model uses an encoder-decoder architecture with a shared latent space, where the encoder maps input captions to a continuous vector representation.
• A generator network produces synthetic captions from a random noise vector and, in the conditional variant, from a class label (e.g., 'normal', 'cardiomegaly').
• A discriminator is trained to distinguish between real captions (from the dataset) and generated ones, with both real and fake samples conditioned on labels.
• The training combines adversarial loss (using Earth-Mover distance for stability) with autoencoding reconstruction loss to regularize the latent space.
• Conditional ARAE extends the framework by injecting class labels into both generator and discriminator, enabling controlled text generation.
• The model is trained on a chest X-ray caption dataset from Indiana University, with vocabulary filtering and label binarization into three diagnostic categories.

Experimental results

Research questions

• RQ1 Can an adversarially regularized autoencoder generate medically plausible, continuous representations of clinical text with high linguistic quality?
• RQ2 Does conditioning the text generation on diagnostic labels improve the coherence and relevance of generated captions compared to unconditional generation?
• RQ3 Can the ARAE framework outperform a standard LSTM language model in terms of perplexity on a medical captioning task?
• RQ4 How robust is the model to class imbalance and overlapping semantic content across diagnostic categories in clinical reports?
• RQ5 To what extent can the learned continuous representations support downstream NLP tasks requiring smooth, differentiable input?

Key findings

• The conditional ARAE achieves a perplexity of 125.4, significantly outperforming the LSTM baseline (150.0) and the unconditional ARAE (148.4).
• Generated captions exhibit syntactic correctness and medical terminology consistency, with qualitative examples showing plausible clinical descriptions for each diagnostic class.
• The model successfully learns to generate diverse, realistic reports conditioned on labels, even for rare conditions like cardiomegaly, though performance is slightly degraded due to data scarcity.
• The adversarial training setup with Earth-Mover distance loss stabilizes training and enables effective distribution learning despite the discrete nature of text.
• The demo interface confirms the model’s ability to generate varied, label-conditional outputs and to distinguish real from fake captions via a discriminator game.
• Despite training instability due to multiple loss functions and high model complexity, the model converges sufficiently to produce clinically plausible text.