[Paper Review] Deep Rib Fracture Instance Segmentation and Classification from CT on the RibFrac Challenge

Jiancheng Yang, Rui Shi+23 more|arXiv (Cornell University)|Feb 14, 2024

Trauma Management and Diagnosis4 citations

TL;DR

This paper presents FracNet+, a deep learning framework for 3D instance segmentation and classification of rib fractures in CT scans using the RibFrac Challenge dataset. By integrating point-based rib segmentation and large-scale pre-trained 3D networks, it achieves state-of-the-art performance in fracture detection, outperforming human experts in some cases, though classification of subtle types like buckle fractures remains challenging.

ABSTRACT

Rib fractures are a common and potentially severe injury that can be challenging and labor-intensive to detect in CT scans. While there have been efforts to address this field, the lack of large-scale annotated datasets and evaluation benchmarks has hindered the development and validation of deep learning algorithms. To address this issue, the RibFrac Challenge was introduced, providing a benchmark dataset of over 5,000 rib fractures from 660 CT scans, with voxel-level instance mask annotations and diagnosis labels for four clinical categories (buckle, nondisplaced, displaced, or segmental). The challenge includes two tracks: a detection (instance segmentation) track evaluated by an FROC-style metric and a classification track evaluated by an F1-style metric. During the MICCAI 2020 challenge period, 243 results were evaluated, and seven teams were invited to participate in the challenge summary. The analysis revealed that several top rib fracture detection solutions achieved performance comparable or even better than human experts. Nevertheless, the current rib fracture classification solutions are hardly clinically applicable, which can be an interesting area in the future. As an active benchmark and research resource, the data and online evaluation of the RibFrac Challenge are available at the challenge website. As an independent contribution, we have also extended our previous internal baseline by incorporating recent advancements in large-scale pretrained networks and point-based rib segmentation techniques. The resulting FracNet+ demonstrates competitive performance in rib fracture detection, which lays a foundation for further research and development in AI-assisted rib fracture detection and diagnosis.

Motivation & Objective

• To address the clinical challenge of missed rib fractures in CT scans, particularly due to complex rib anatomy and subtle fracture types.
• To develop a robust deep learning framework for 3D instance segmentation and classification of rib fractures in volumetric CT data.
• To establish a benchmark for AI-assisted rib fracture diagnosis through the RibFrac Challenge, providing a large-scale, annotated dataset with voxel-level masks and clinical labels.
• To improve detection accuracy and reduce interpretation time compared to manual reading, especially for complex or overlooked fracture types.
• To identify key technical hurdles in classification, such as geometric complexity and class imbalance, and guide future methodological improvements.

Proposed method

• The method employs a point-based rib segmentation technique to extract anatomical structures and guide subsequent fracture detection.
• It integrates large-scale pre-trained 3D convolutional neural networks (e.g., MONAI-based architectures) fine-tuned on the RibFrac dataset for improved feature learning.
• A 3D instance segmentation head is used to localize and segment individual rib fractures across multiple CT slices with voxel-level precision.
• The classification head predicts four clinical categories: buckle (BK), nondisplaced (ND), displaced (DP), and segmental (SG) fractures.
• The framework is trained end-to-end using a multi-task loss combining detection and classification objectives, with data augmentation to improve robustness.
• Post-challenge enhancements include improved data sampling and loss weighting to address class imbalance and geometric variability.

Figure 1: Illustration of two tracks in the RibFrac Challenge. In the rib fracture detection track, participants submit the 3D instance segmentation mask for each fracture to detect their location and extent. In the rib fracture classification track, participants submit the label for each fracture t

Experimental results

Research questions

• RQ1 Can deep learning models achieve human-level or better performance in detecting rib fractures across diverse CT scan volumes?
• RQ2 How do geometric complexity and variable rib orientation affect the performance of 3D CNNs in rib fracture detection?
• RQ3 Why do current models struggle with classifying subtle fracture types like buckle and segmental fractures despite strong detection performance?
• RQ4 To what extent does incorporating point-based rib segmentation improve the accuracy and robustness of fracture detection?
• RQ5 Can unified frameworks that integrate rib labeling, centerline extraction, and fracture diagnosis outperform task-specific pipelines?

Key findings

• Several top-performing models in the RibFrac Challenge achieved detection performance comparable to or exceeding that of human radiologists, particularly in identifying complex or missed fractures.
• The prediction-aware F1 score for buckle fractures showed significant improvement, indicating reduced false negatives, though classification remains difficult due to subtle appearance.
• Segmental fractures were poorly detected due to reliance on local patches and lack of long-range structural context, highlighting a key limitation in current methods.
• The distribution of fracture types in the dataset is imbalanced, with fewer segmental and buckle fractures, which complicates model generalization and increases error rates.
• The proposed FracNet+ framework demonstrated competitive performance in detection by leveraging pre-trained 3D networks and point-based rib segmentation, setting a strong baseline for future work.

Figure 2: Statistics and sample visualization of the RibFrac Challenge dataset. A) Histograms that display the distribution of the number of slices per scan (left), the number of rib fractures per scan (middle), and the volumetric size of each individual rib fracture (right). B) Visualization of fou