[Paper Review] TorchXRayVision: A library of chest X-ray datasets and models

Joseph Cohen, Joseph D. Viviano+9 more|arXiv (Cornell University)|Oct 31, 2021

COVID-19 diagnosis using AI30 references37 citations

TL;DR

TorchXRayVision is an open-source Python library that standardizes access to multiple chest X-ray datasets and models, enabling reproducible baselines, feature extraction, and covariate-shift analysis across datasets.

ABSTRACT

TorchXRayVision is an open source software library for working with chest X-ray datasets and deep learning models. It provides a common interface and common pre-processing chain for a wide set of publicly available chest X-ray datasets. In addition, a number of classification and representation learning models with different architectures, trained on different data combinations, are available through the library to serve as baselines or feature extractors.

Motivation & Objective

• Provide a reusable framework with a common interface for chest X-ray datasets and models to enable reproducible research and consistent baselines.
• Offer pre-trained models and feature extractors for rapid baseline comparisons and transfer learning.
• Facilitate model evaluation, generalization studies, and analysis of failures via standardized data handling and processing.

Proposed method

• Object-oriented design with clear separation between datasets and models.
• Pre-trained core classifiers and baseline classifiers with a uniform input/output interface.
• Dataset classes with common fields (pathologies, labels, and metadata) and tools for relabeling, filtering, and merging.
• Image preprocessing aligning inputs to model training resolutions and pixel value ranges [-1024, 1024].
• Support for autoencoders and feature extraction to generate latent representations for downstream tasks.
• Covariate-shift simulation tools to study generalization across datasets.

Experimental results

Research questions

• RQ1 How well do chest X-ray models generalize across diverse public datasets when swapped via a common interface?
• RQ2 Can pre-trained models serve as reliable baselines or feature extractors across different chest X-ray tasks?
• RQ3 What is the impact of covariate shift on model performance and attribution, and how can it be studied or mitigated using TorchXRayVision?
• RQ4 How can a unified dataset protocol enable reproducible model evaluation, data fusion, and subset analyses across multiple datasets?

Key findings

• Provides a unified API to swap datasets and models with automatic input resizing and standard preprocessing.
• Includes core classifiers and baseline models with easy weight specification and downloadable pre-trained weights.
• Supports feature extraction and visualization of representations (e.g., UMAP) across models and datasets.
• Offers autoencoders for unsupervised representations and reconstruction-based analysis.
• Implements covariate-shift tools to simulate distribution shifts between datasets for robustness testing.
• Includes utilities for dataset relabeling, merging, subsetting, and handling pathology/semantic masks for segmentation work.