[Paper Review] Rethinking Medical Report Generation: Disease Revealing Enhancement with Knowledge Graph

Yixin Wang, Zihao Lin+1 more|arXiv (Cornell University)|Jul 24, 2023

Topic ModelingComputer Science3 citations

TL;DR

This paper proposes a two-stage medical report generation framework enhanced by a knowledge graph (KG) to address long-tailed disease distribution in chest X-ray reports. By constructing a comprehensive KG with 137 disease types and applying KG-guided data augmentation, the method improves clinical relevance through a novel metric, diverse sensitivity (DS), achieving a 25% relative improvement in DS (0.1902) over baseline models, significantly enhancing generation of rare disease mentions.

ABSTRACT

Knowledge Graph (KG) plays a crucial role in Medical Report Generation (MRG) because it reveals the relations among diseases and thus can be utilized to guide the generation process. However, constructing a comprehensive KG is labor-intensive and its applications on the MRG process are under-explored. In this study, we establish a complete KG on chest X-ray imaging that includes 137 types of diseases and abnormalities. Based on this KG, we find that the current MRG data sets exhibit a long-tailed problem in disease distribution. To mitigate this problem, we introduce a novel augmentation strategy that enhances the representation of disease types in the tail-end of the distribution. We further design a two-stage MRG approach, where a classifier is first trained to detect whether the input images exhibit any abnormalities. The classified images are then independently fed into two transformer-based generators, namely, ``disease-specific generator" and ``disease-free generator" to generate the corresponding reports. To enhance the clinical evaluation of whether the generated reports correctly describe the diseases appearing in the input image, we propose diverse sensitivity (DS), a new metric that checks whether generated diseases match ground truth and measures the diversity of all generated diseases. Results show that the proposed two-stage generation framework and augmentation strategies improve DS by a considerable margin, indicating a notable reduction in the long-tailed problem associated with under-represented diseases.

Motivation & Objective

• To address the long-tailed distribution of diseases in medical report generation datasets, where rare diseases are under-represented.
• To improve the clinical relevance of generated reports by ensuring accurate and diverse mention of disease keywords, especially for rare conditions.
• To develop a knowledge graph (KG) with 137 chest X-ray disease types to model disease relationships and support data augmentation and evaluation.
• To propose a new evaluation metric, diverse sensitivity (DS), that measures both correctness and diversity of disease mentions in generated reports.
• To demonstrate that training and evaluation based on clinical relevance—guided by the KG—outperform standard N-gram metrics like BLEU in real-world utility.

Proposed method

• A comprehensive knowledge graph (KG) is constructed from two public datasets (IU-Xray and MIMIC-CXR), integrating 137 disease types across 8 categories to model disease relationships.
• A two-stage generation framework is designed: first, a classifier detects whether an image contains any abnormalities; second, two separate transformer-based generators produce reports—'disease-specific' or 'disease-free'—based on the classification.
• A novel data augmentation strategy is introduced, using the KG to generate synthetic training samples for low-frequency disease classes, balancing the long-tailed distribution.
• The diverse sensitivity (DS) metric is proposed to evaluate model performance by measuring both the recall of ground-truth disease keywords and the diversity of generated disease mentions.
• The training and model selection are optimized based on the DS score, not BLEU, to prioritize clinical accuracy over n-gram matching.
• The framework is evaluated on the IU-Xray dataset, with hyperparameters and settings aligned with the R2Gen baseline for fair comparison.

Figure 1: Illustration of counts of labeled sentences and disease keywords in IU-Xray. Part (a) shows the count of sentences that have common diseases (d_com), uncommon diseases (d_tail), or do not have diseases (d_free). Part (b) shows parts of distributions of diseases and abnormalities in origina

Experimental results

Research questions

• RQ1 How does the long-tailed distribution of disease frequencies in existing medical report datasets affect model performance, particularly for rare diseases?
• RQ2 Can a knowledge graph be effectively used to guide data augmentation and improve the representation of low-frequency disease classes in medical report generation?
• RQ3 Does a two-stage generation framework, separating disease-specific and disease-free report generation, lead to better clinical relevance than end-to-end models?
• RQ4 Can a new evaluation metric, diverse sensitivity (DS), better reflect clinical utility than standard N-gram metrics like BLEU in medical report generation?
• RQ5 To what extent does training based on DS improve the generation of rare disease mentions compared to training based on BLEU?

Key findings

• The proposed two-stage generation framework with KG-based augmentation achieved a DS score of 0.1902, a significant improvement over R2Gen’s 0.1523, indicating better detection and diversity of disease mentions.
• The DOR score of 0.5138 for the proposed method outperforms R2Gen’s 0.2911, showing stronger balance between disease detection and report quality.
• When the augmentation strategy was removed, the DS score dropped to 0.1634, confirming the critical role of data balancing in improving rare disease generation.
• The disease-specific-only generator achieved a high DS of 0.1955 but scored zero on DOR due to its inability to generate normal reports, highlighting the clinical limitation of such models.
• A BLEU-optimized R2Gen model achieved a high BLEU score but near-zero DS (0.0324) and sensitivity (0.0186), demonstrating that standard metrics fail to reflect clinical accuracy.
• Qualitative analysis confirmed that the proposed method successfully generated clinically relevant descriptions of rare abnormalities like 'interstitial opacity' and 'edema', aligning with KG-annotated keywords.

Figure 2: An illustration of our proposed knowledge graph, which contains “normal” and 8 disease categories including 7 organs and an “other” category. Each category further branches out into its corresponding specific diseases.