[Paper Review] Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification

Qingji Guan, Yaping Huang+4 more|arXiv (Cornell University)|Jan 30, 2018

COVID-19 diagnosis using AIMedicine31 references241 citations

TL;DR

AG-CNN introduces a three-branch architecture (global, local, fusion) that uses attention-guided local regions cropped from global CXR images to boost thorax disease classification on ChestX-ray14, achieving state-of-the-art AUC.

ABSTRACT

This paper considers the task of thorax disease classification on chest X-ray images. Existing methods generally use the global image as input for network learning. Such a strategy is limited in two aspects. 1) A thorax disease usually happens in (small) localized areas which are disease specific. Training CNNs using global image may be affected by the (excessive) irrelevant noisy areas. 2) Due to the poor alignment of some CXR images, the existence of irregular borders hinders the network performance. In this paper, we address the above problems by proposing a three-branch attention guided convolution neural network (AG-CNN). AG-CNN 1) learns from disease-specific regions to avoid noise and improve alignment, 2) also integrates a global branch to compensate the lost discriminative cues by local branch. Specifically, we first learn a global CNN branch using global images. Then, guided by the attention heat map generated from the global branch, we inference a mask to crop a discriminative region from the global image. The local region is used for training a local CNN branch. Lastly, we concatenate the last pooling layers of both the global and local branches for fine-tuning the fusion branch. The Comprehensive experiment is conducted on the ChestX-ray14 dataset. We first report a strong global baseline producing an average AUC of 0.841 with ResNet-50 as backbone. After combining the local cues with the global information, AG-CNN improves the average AUC to 0.868. While DenseNet-121 is used, the average AUC achieves 0.871, which is a new state of the art in the community.

Motivation & Objective

• Motivation to improve thorax disease classification by focusing on disease-specific local regions rather than the whole image.
• Address misalignment and noisy background in chest X-ray analysis.
• Propose an attention-guided, three-branch architecture to fuse global and local cues for better accuracy.

Proposed method

• Propose AG-CNN with three branches: global, local, and fusion, operating on global images and attention-cropped local regions.
• Generate an attention heat map from the global branch by computing the max over channels of the absolute activations in the last convolutional layer (H_g).
• Infer a binary mask M by thresholding H_g with a threshold tau, extract the maximum connected region, and crop I_c from I as the local input.
• Train global and local branches separately, then fuse their Pool5 features for final classification in a fusion branch.
• Use a three-stage training protocol: Stage I (train global branch), Stage II (crop local region and train local branch), Stage III (train fusion branch with concatenated Pool5 features).
• Evaluate on ChestX-ray14 with 14 pathologies plus No Finding using a multi-label setup (C=15).

Experimental results

Research questions

• RQ1 Can attention-guided local region cropping improve thorax disease classification over global-image baselines?
• RQ2 Do global and local branches provide complementary information that, when fused, yield higher accuracy than either branch alone?
• RQ3 How sensitive is the method to the local-region threshold tau and to backbone choice (ResNet-50 vs DenseNet-121)?
• RQ4 Is the proposed approach robust to misalignment and background noise in chest X-ray images?

Key findings

• Global baseline (ResNet-50) achieves average AUC of 0.841 on ChestX-ray14.
• AG-CNN with global + local fusion improves average AUC to 0.868 (ResNet-50) and 0.871 (DenseNet-121), achieving state-of-the-art.
• Local branch alone yields competitive but lower average AUC (~0.817–0.810) than the global baseline, indicating the local crop helps when fused.
• Fusion branch consistently outperforms both individual branches across the 14 pathologies for both backbones.
• The approach is relatively robust to parameter changes, with tau around 0.7 providing strong performance and fusion providing gains over either branch alone.
• Most notable gains are seen in small-lesion categories (e.g., Nodule) where local attention helps reduce background noise.