[Paper Review] Vertebrae Detection and Localization in CT with Two-Stage CNNs and Dense Annotations

James McCouat, Ben Glocker|arXiv (Cornell University)|Oct 14, 2019

Medical Imaging and Analysis11 references4 citations

TL;DR

This paper proposes a two-stage CNN framework for vertebrae detection and localization in CT scans, using 3D dense labeling for initial detection and 2D continuous regression for vertebrae identification via L1 loss to preserve anatomical ordering. The method improves state-of-the-art mean localization accuracy by 0.87mm on a pathological spine CT benchmark.

ABSTRACT

We propose a new, two-stage approach to the vertebrae centroid detection and localization problem. The first stage detects where the vertebrae appear in the scan using 3D samples, the second identifies the specific vertebrae within that region-of-interest using 2D slices. Our solution utilizes new techniques to improve the accuracy of the algorithm such as a revised approach to dense labelling from sparse centroid annotations and usage of large anisotropic kernels in the base level of a U-net architecture to maximize the receptive field. Our method improves the state-of-the-art's mean localization accuracy by 0.87mm on a publicly available spine CT benchmark.

Motivation & Objective

• To address the challenge of accurate vertebrae localization in arbitrary Field-of-View (FoV) CT scans, especially in pathological cases with missing or abnormal anatomy.
• To improve upon existing methods that rely on regression or sparse labeling by using dense annotations to enhance contextual understanding.
• To develop a lightweight, fast inference method that captures both short- and long-range anatomical context for improved localization.
• To explicitly model the anatomical ordering of vertebrae using continuous regression instead of discrete classification.
• To reduce localization error in clinical-relevant scenarios where scans are limited in coverage and contain artifacts.

Proposed method

• A two-stage approach: first, a 3D U-Net with large anisotropic kernels detects vertebral bodies in 3D patches (64×64×80), using weighted categorical cross-entropy loss with class imbalance correction.
• Second, a 2D U-Net predicts continuous values per pixel representing the vertebrae index (e.g., C4 = 4), using L1 loss to preserve ordering and enable regression-based refinement.
• Dense labeling is generated from sparse centroid annotations via a revised method that ensures spatial consistency and coverage across the spine.
• Final predictions are obtained by element-wise multiplication of detection and identification outputs, followed by centroid aggregation from peak locations.
• The detection model uses 5 random 3D crops per scan, while the identification model uses large 2D slices to capture long-range context.
• The method avoids iterative inference (unlike RNN-based SOTA) and achieves fast inference (~40 seconds per scan).

Experimental results

Research questions

• RQ1 Can a two-stage CNN framework with dense annotations improve localization accuracy over regression-based or classification-based methods?
• RQ2 Does modeling vertebrae ordering via continuous regression (L1 loss) lead to better localization than discrete classification?
• RQ3 Can large receptive fields via anisotropic kernels in a 3D U-Net improve detection of sparse or pathological vertebral regions?
• RQ4 How does the performance of the method vary across different spinal regions (cervical, thoracic, lumbar)?
• RQ5 Can the method generalize to scans with limited FoV and severe artifacts without explicit shape priors?

Key findings

• The method improves mean localization error by 0.87mm compared to the SOTA method by Liao et al. [6], achieving a mean error of 5.60mm on the full test set.
• The Id rate (percentage of predictions within 20mm of the true centroid) is 85.8%, slightly lower than Liao et al.’s 88.3%, suggesting trade-offs in robustness for thoracic vertebrae.
• The method achieves the best performance on cervical vertebrae (3.93mm mean error), outperforming SOTA in this region.
• Thoracic vertebrae show the highest error (6.61mm mean), with T6 and T7 being under-represented (80 and 82 scans), indicating data imbalance issues.
• Training takes 11 hours for detection and 7 hours for identification, with inference completed in ~40 seconds per scan, demonstrating high efficiency.
• The method is robust to artifacts and missing anatomy due to contextual modeling, though performance degrades when only thoracic vertebrae are visible and no end points are present.