[Paper Review] CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formats

Pierre Chambon, Jean-Benoit Delbrouck+7 more|arXiv (Cornell University)|May 29, 2024

Radiomics and Machine Learning in Medical ImagingMedicine3 citations

TL;DR

CheXpert Plus significantly enhances the CheXpert chest X-ray dataset by integrating de-identified radiology reports, patient demographics, pathology labels, RadGraph annotations, and multi-format images (DICOM and PNG), enabling large-scale, fair, and robust AI research in radiology. The dataset includes 36 million text tokens, 1 million anonymized PHI spans, and supports cross-institutional training, with models and data publicly released for non-commercial research use.

ABSTRACT

Since the release of the original CheXpert paper five years ago, CheXpert has become one of the most widely used and cited clinical AI datasets. The emergence of vision language models has sparked an increase in demands for sharing reports linked to CheXpert images, along with a growing interest among AI fairness researchers in obtaining demographic data. To address this, CheXpert Plus serves as a new collection of radiology data sources, made publicly available to enhance the scaling, performance, robustness, and fairness of models for all subsequent machine learning tasks in the field of radiology. CheXpert Plus is the largest text dataset publicly released in radiology, with a total of 36 million text tokens, including 13 million impression tokens. To the best of our knowledge, it represents the largest text de-identification effort in radiology, with almost 1 million PHI spans anonymized. It is only the second time that a large-scale English paired dataset has been released in radiology, thereby enabling, for the first time, cross-institution training at scale. All reports are paired with high-quality images in DICOM format, along with numerous image and patient metadata covering various clinical and socio-economic groups, as well as many pathology labels and RadGraph annotations. We hope this dataset will boost research for AI models that can further assist radiologists and help improve medical care. Data is available at the following URL: https://stanfordaimi.azurewebsites.net/datasets/5158c524-d3ab-4e02-96e9-6ee9efc110a1 Models are available at the following URL: https://github.com/Stanford-AIMI/chexpert-plus

Motivation & Objective

• To address the growing demand for paired image-text data in radiology AI by augmenting CheXpert with radiology reports and patient demographics.
• To enable cross-institutional training and improve model robustness by releasing a large-scale, English-language, multi-source dataset.
• To support fairness research by including clinical and socio-economic demographic data to mitigate bias in AI models.
• To provide pre-annotated RadGraph and pathology labels to streamline downstream tasks such as report generation and classification.
• To release a comprehensive set of models trained on the augmented data for key radiology tasks, including text-to-image and text-to-text generation.

Proposed method

• The dataset was constructed by collecting and pairing high-quality DICOM and PNG images from the original CheXpert dataset with corresponding radiology reports.
• Reports were parsed into subsections (Findings and Impression) and de-identified using a hybrid pipeline combining rule-based NLP and LLM-based anonymization, followed by human review by a board-certified radiologist.
• Patient demographics, including sex, ethnicity, age, and insurance status, were integrated to support fairness and bias analysis.
• Pathology labels were automatically extracted from reports using NLP techniques, and RadGraph annotations were computed for both Findings and Impression sections.
• A rigorous four-step de-identification process ensured all 1 million PHI spans were replaced with synthetic equivalents while preserving report structure and meaning.
• The dataset was released under a research-use-only agreement, with models trained on the data made available for tasks like report generation, summarization, and classification.

Experimental results

Research questions

• RQ1 How can a large-scale, de-identified, multi-source radiology dataset improve the performance and robustness of vision-language models in chest X-ray analysis?
• RQ2 To what extent can inclusion of demographic and socio-economic data in a radiology dataset reduce bias in AI model predictions?
• RQ3 Can the availability of paired reports and images in multiple formats (DICOM and PNG) enable effective cross-institutional training and improve model generalization?
• RQ4 How effective is a hybrid automated and human-reviewed de-identification pipeline in preserving data utility while ensuring patient privacy in radiology reports?
• RQ5 What impact does the inclusion of RadGraph-structured annotations have on downstream tasks such as report generation and medical concept extraction?

Key findings

• CheXpert Plus is the largest publicly released text dataset in radiology, containing 36 million text tokens, including 13 million impression tokens, and is the first such dataset to include de-identified reports at scale.
• The dataset includes nearly 1 million anonymized PHI spans, representing the largest de-identification effort in radiology to date, with a hybrid automated and human-reviewed process ensuring privacy.
• It is only the second large-scale English paired image-text dataset in radiology, enabling cross-institutional training and improving model robustness across diverse clinical settings.
• The dataset provides 14 automatically extracted pathology labels and pre-computed RadGraph annotations for both Findings and Impression sections, streamlining downstream NLP and vision-language tasks.
• The de-identification pipeline successfully replaced all true PHI spans with synthetic ones, with only 0.01% of PHI spans remaining partially missed after the final validation step.
• The release includes a suite of pre-trained models for key radiology tasks, such as text-to-image generation and text summarization, accelerating research in medical AI.