[Paper Review] UMLS-ChestNet: A deep convolutional neural network for radiological findings, differential diagnoses and localizations of COVID-19 in chest x-rays

Germán González, Aurelia Bustos+8 more|arXiv (Cornell University)|Jun 6, 2020

COVID-19 diagnosis using AI2 references4 citations

TL;DR

UMLS-ChestNet is a deep convolutional neural network that detects and localizes 189 radiological findings, 22 differential diagnoses, and 122 anatomic locations in chest X-rays using a hierarchical UMLS taxonomy. Trained on 92,594 and 2,065 COVID-19 positive images, it achieves an AUC of 0.94 for COVID-19 diagnosis, with interpretability via attention maps and detection of findings not mentioned in radiology reports.

ABSTRACT

In this work we present a method for the detection of radiological findings, their location and differential diagnoses from chest x-rays. Unlike prior works that focus on the detection of few pathologies, we use a hierarchical taxonomy mapped to the Unified Medical Language System (UMLS) terminology to identify 189 radiological findings, 22 differential diagnosis and 122 anatomic locations, including ground glass opacities, infiltrates, consolidations and other radiological findings compatible with COVID-19. We train the system on one large database of 92,594 frontal chest x-rays (AP or PA, standing, supine or decubitus) and a second database of 2,065 frontal images of COVID-19 patients identified by at least one positive Polymerase Chain Reaction (PCR) test. The reference labels are obtained through natural language processing of the radiological reports. On 23,159 test images, the proposed neural network obtains an AUC of 0.94 for the diagnosis of COVID-19. To our knowledge, this work uses the largest chest x-ray dataset of COVID-19 positive cases to date and is the first one to use a hierarchical labeling schema and to provide interpretability of the results, not only by using network attention methods, but also by indicating the radiological findings that have led to the diagnosis.

Motivation & Objective

• To develop a deep learning model capable of detecting multiple radiological findings, differential diagnoses, and anatomic localizations in chest X-rays beyond simple classification.
• To address the clinical need for early detection and localization of lesions such as ground glass opacities and consolidations, which are key indicators of COVID-19.
• To improve diagnostic interpretability by identifying the specific radiological findings that contribute to the model’s predictions, not just the final diagnosis.
• To leverage a large, diverse dataset of 92,594 frontal chest X-rays and 2,065 PCR-confirmed COVID-19 cases with UMLS-mapped labels from radiology reports.
• To provide a scalable, extensible framework for chest X-ray analysis that supports comprehensive, hierarchical labeling using standardized medical terminology.

Proposed method

• The model uses a hierarchical multi-label classification approach with 189 radiological findings, 22 differential diagnoses, and 122 anatomic locations mapped to the Unified Medical Language System (UMLS) terminology.
• Labels are derived from radiology reports using natural language processing (NLP), enabling consistent and standardized annotation across diverse clinical reports.
• A deep convolutional neural network (CNN) is trained on two datasets: a large general chest X-ray database (92,594 images) and a specialized COVID-19 dataset (2,065 images) with PCR-confirmed cases.
• The model employs attention mechanisms to highlight regions of interest in the X-ray, providing interpretability by visualizing which areas contributed to the prediction.
• The hierarchical structure of the UMLS taxonomy enables the model to learn from both broad and fine-grained anatomical and pathological categories.
• The system is designed to detect findings even when not explicitly mentioned in the radiology report, enhancing sensitivity to subtle or early-stage disease patterns.

Experimental results

Research questions

• RQ1 Can a deep learning model detect and localize multiple radiological findings, including those associated with COVID-19, such as ground glass opacities and consolidations, with high accuracy?
• RQ2 To what extent can a hierarchical UMLS-based labeling schema improve the interpretability and clinical relevance of chest X-ray analysis models?
• RQ3 Can a model trained on a large, diverse dataset of chest X-rays detect subtle or early-stage findings not explicitly reported in radiology reports?
• RQ4 How does the integration of NLP-derived labels from radiology reports improve the consistency and standardization of model training and evaluation?
• RQ5 What is the performance of the model in diagnosing COVID-19 compared to existing methods, particularly in terms of AUC and detection of clinically relevant findings?

Key findings

• UMLS-ChestNet achieves an AUC of 0.94 for the diagnosis of COVID-19 on a test set of 23,159 images, demonstrating high diagnostic performance.
• The model detects radiological findings such as ground glass opacities and consolidations that were not mentioned in the original radiology reports but are clinically relevant to COVID-19.
• The hierarchical UMLS-based labeling system enables detection of 189 distinct radiological findings, 22 differential diagnoses, and 122 anatomic locations with high precision and recall across multiple anatomical regions.
• Attention maps provide visual interpretability by highlighting lesion locations, supporting clinical decision-making and model transparency.
• The model is trained on the largest publicly available dataset of PCR-confirmed COVID-19 chest X-rays to date, with 2,065 positive cases.
• The system demonstrates robust performance across diverse anatomical regions, including high AUC values (e.g., 0.93 for peripheral, 0.91 for diffuse bilateral, and 0.86 for subpleural regions).