[Paper Review] Learning to diagnose from scratch by exploiting dependencies among labels

Yao Li, Eric Poblenz+4 more|arXiv (Cornell University)|Oct 28, 2017

COVID-19 diagnosis using AIMedicine21 references195 citations

TL;DR

This paper trains chest X-ray diagnosis models from scratch, using a DenseNet encoder and LSTM-based decoder to model dependencies among 14 abnormalities, achieving state-of-the-art results without pre-training on ImageNet.

ABSTRACT

The field of medical diagnostics contains a wealth of challenges which closely resemble classical machine learning problems; practical constraints, however, complicate the translation of these endpoints naively into classical architectures. Many tasks in radiology, for example, are largely problems of multi-label classification wherein medical images are interpreted to indicate multiple present or suspected pathologies. Clinical settings drive the necessity for high accuracy simultaneously across a multitude of pathological outcomes and greatly limit the utility of tools which consider only a subset. This issue is exacerbated by a general scarcity of training data and maximizes the need to extract clinically relevant features from available samples -- ideally without the use of pre-trained models which may carry forward undesirable biases from tangentially related tasks. We present and evaluate a partial solution to these constraints in using LSTMs to leverage interdependencies among target labels in predicting 14 pathologic patterns from chest x-rays and establish state of the art results on the largest publicly available chest x-ray dataset from the NIH without pre-training. Furthermore, we propose and discuss alternative evaluation metrics and their relevance in clinical practice.

Motivation & Objective

• Address multi-label chest X-ray diagnosis with many correlated abnormalities.
• Eliminate reliance on pre-training to reduce bias and improve clinical relevance.
• Leverage inter-label dependencies to improve predictive performance across all targets.
• Introduce clinically meaningful evaluation metrics beyond traditional BLEU-like scores.

Proposed method

• Use a densely connected DenseNet-like image encoder to process high-resolution chest X-rays.
• Predict multiple abnormalities with a recurrent neural network decoder to capture label dependencies.
• Apply a sigmoid-based decoding at each step to allow presence/absence of each abnormality without a predefined stop token.
• Train end-to-end from scratch without ImageNet pre-training.
• Experiment with two dependency-aware decoding variants and compare orderings of label prediction.

Experimental results

Research questions

• RQ1 Can a label-dependency aware decoder improve multi-label chest X-ray diagnosis when trained from scratch?
• RQ2 What is the impact of including conditional dependencies among labels on predictive performance?
• RQ3 Do different label orderings in the dependency modeling affect performance when enough training data is available?

Key findings

• A baseline model with independent labels trained from scratch outperforms the pre-trained state-of-the-art.
• Modeling inter-label dependencies yields improvements across multiple metrics (NLL, DICE, PESS, PCSS).
• Ordering of label dependencies has marginal effect when the model is well trained.
• The approach achieves higher per-abnormality AUCs compared to the prior method on the ChestX-ray8 dataset.