[Paper Review] An Effective Transformer-based Solution for RSNA Intracranial Hemorrhage Detection Competition

Fangxin Shang, Siqi Wang+2 more|arXiv (Cornell University)|May 16, 2022

Intracerebral and Subarachnoid Hemorrhage Research4 citations

TL;DR

This paper presents a single, end-to-end Transformer-based model for intracranial hemorrhage detection that outperforms the RSNA 2019 competition winner using only 1/4 the parameters and 10% of the FLOPs. By leveraging Swin-Transformer for intra-slice features and a two-layer sequential Transformer for inter-slice modeling, combined with FixMatch semi-supervised learning, the method achieves a private test log-loss of 0.04345 with a three-model ensemble and 0.04368 with a single model—surpassing the winning solution’s 0.04383.

ABSTRACT

We present an effective method for Intracranial Hemorrhage Detection (IHD) which exceeds the performance of the winner solution in RSNA-IHD competition (2019). Meanwhile, our model only takes quarter parameters and ten percent FLOPs compared to the winner's solution. The IHD task needs to predict the hemorrhage category of each slice for the input brain CT. We review the top-5 solutions for the IHD competition held by the Radiological Society of North America(RSNA) in 2019. Nearly all the top solutions rely on 2D convolutional networks and sequential models (Bidirectional GRU or LSTM) to extract intra-slice and inter-slice features, respectively. All the top solutions enhance the performance by leveraging the model ensemble, and the model number varies from 7 to 31. In the past years, since much progress has been made in the computer vision regime especially Transformer-based models, we introduce the Transformer-based techniques to extract the features in both intra-slice and inter-slice views for IHD tasks. Additionally, a semi-supervised method is embedded into our workflow to further improve the performance. The code is available in the manuscript.

Motivation & Objective

• To develop a more efficient and accurate model for intracranial hemorrhage detection than the top-performing solution in the RSNA 2019 competition.
• To address the limitations of decoupled intra- and inter-slice feature learning in existing 2D CNN and RNN-based models, which suffer from error accumulation and lack end-to-end optimization.
• To reduce reliance on large-scale model ensembles by integrating semi-supervised learning, thereby enabling clinical deployment with lower computational cost.
• To explore the effectiveness of modern Transformer architectures in medical image sequence modeling for multi-label classification tasks.

Proposed method

• The intra-slice features are extracted using a Swin-Transformer, which enables high memory efficiency and local-global attention within each CT slice.
• Inter-slice dependencies are modeled using a two-layer sequential Transformer that processes the sequence of intra-slice features in order.
• A residual connection and post-normalization (PN) are applied in the Swin-Transformer to stabilize training and improve gradient flow.
• The model employs a deep-supervision schema (DS) with auxiliary classifiers to accelerate convergence during training.
• FixMatch semi-supervised learning is applied by generating pseudo-labels for unlabeled data and applying strong augmentations to them, improving generalization.
• Model ensemble is used strategically—only three models are combined, with weights assigned based on private leaderboard rank, to boost performance without excessive computational cost.

Experimental results

Research questions

• RQ1 Can a single, end-to-end Transformer-based model outperform the ensemble of multiple 2D CNN and RNN models used in the top RSNA 2019 solutions?
• RQ2 Does replacing the traditional CNN-RNN pipeline with a Transformer-based architecture reduce error accumulation and improve performance?
• RQ3 Can semi-supervised learning via FixMatch effectively improve model accuracy while reducing the need for large model ensembles in medical image classification?
• RQ4 To what extent do architectural choices like post-normalization, dynamic weighting, and data augmentation contribute to performance gains in this task?
• RQ5 Is explicit modeling of the logical relationship between hemorrhage categories (e.g., 'any' as max of others) necessary or beneficial?

Key findings

• The proposed single-model solution achieved a private test log-loss of 0.04368, outperforming the first-place solution in the RSNA 2019 competition, which had a log-loss of 0.04383.
• With a three-model ensemble using FixMatch pseudo-labels, the method achieved a private log-loss of 0.04345, further surpassing the competition winner.
• The model uses only 86M parameters and 15G FLOPs per inference, representing 25% of the parameters and 10% of the FLOPs of the winning solution (330M parameters, 180G FLOPs).
• Ablation studies confirmed that each component—RBE, DS, PN, DW, SW, and FM—contributed incrementally to performance, with the full set reducing log-loss from 0.09495 to 0.04368.
• The use of external data (CQ500) and logical reparameterization of the 'any' class did not yield significant performance gains, suggesting the model is already highly optimized.
• The model's end-to-end training allows full gradient backpropagation through both intra- and inter-slice modules, avoiding error accumulation seen in decoupled pipelines.