[Paper Review] ULDor: A Universal Lesion Detector for CT Scans with Pseudo Masks and Hard Negative Example Mining

Youbao Tang, Ke Yan+4 more|arXiv (Cornell University)|Jan 18, 2019

Advanced Neural Network Applications9 references4 citations

TL;DR

ULDor proposes a universal lesion detector for CT scans using pseudo masks and hard negative example mining (HNEM) to improve detection performance. By leveraging RECIST annotations to generate surrogate masks and iteratively refining the model with mined false positives, ULDor achieves 86.21% sensitivity at five false positives per image, outperforming prior methods.

ABSTRACT

Automatic lesion detection from computed tomography (CT) scans is an important task in medical imaging analysis. It is still very challenging due to similar appearances (e.g. intensity and texture) between lesions and other tissues, making it especially difficult to develop a universal lesion detector. Instead of developing a specific-type lesion detector, this work builds a Universal Lesion Detector (ULDor) based on Mask R-CNN, which is able to detect all different kinds of lesions from whole body parts. As a state-of-the-art object detector, Mask R-CNN adds a branch for predicting segmentation masks on each Region of Interest (RoI) to improve the detection performance. However, it is almost impossible to manually annotate a large-scale dataset with pixel-level lesion masks to train the Mask R-CNN for lesion detection. To address this problem, this work constructs a pseudo mask for each lesion region that can be considered as a surrogate of the real mask, based on which the Mask R-CNN is employed for lesion detection. On the other hand, this work proposes a hard negative example mining strategy to reduce the false positives for improving the detection performance. Experimental results on the NIH DeepLesion dataset demonstrate that the ULDor is enhanced using pseudo masks and the proposed hard negative example mining strategy and achieves a sensitivity of 86.21% with five false positives per image.

Motivation & Objective

• To develop a universal lesion detector capable of identifying diverse lesion types across whole-body CT scans.
• To address the challenge of limited pixel-level lesion annotations by constructing pseudo masks from RECIST annotations.
• To reduce false positives in lesion detection using a hard negative example mining (HNEM) strategy.
• To iteratively improve detection performance by retraining the Mask R-CNN model with mined negative examples.
• To achieve state-of-the-art sensitivity while minimizing false positives in clinical-relevant settings.

Proposed method

• Constructs pseudo masks for lesion regions using RECIST annotations as surrogate ground truth for Mask R-CNN training.
• Employs Mask R-CNN with RoIAlign to ensure precise spatial alignment for mask prediction.
• Applies a hard negative example mining (HNEM) strategy to identify and retrain on false positive candidates.
• Iteratively re-trains the Mask R-CNN model using both lesion and mined negative examples to improve feature discrimination.
• Uses a single forward pass for inference during testing, leveraging the final refined model.
• Combines pseudo masks and HNEM in a joint training loop to enhance detection robustness and sensitivity.

Experimental results

Research questions

• RQ1 Can pseudo masks derived from RECIST annotations effectively substitute for real pixel-level masks in training a universal lesion detector?
• RQ2 Does hard negative example mining significantly reduce false positives in lesion detection from CT scans?
• RQ3 Can the combination of pseudo masks and HNEM improve detection sensitivity beyond existing methods?
• RQ4 How does the iterative retraining with mined negative examples affect the ranking of true positives?
• RQ5 What is the trade-off between sensitivity and false positive rate when using pseudo masks and HNEM?

Key findings

• ULDor achieves a sensitivity of 86.21% with five false positives per image, representing an absolute improvement of 5.11% over the prior state-of-the-art method (81.1%).
• The use of pseudo masks increases sensitivity compared to Mask R-CNN without mask prediction, demonstrating the value of segmentation supervision.
• Hard negative example mining reduces false positives, as shown by higher sensitivity at low false positive rates (e.g., 5 FPs/image) compared to baseline models.
• The method outperforms Yan et al. [4] at all false positive levels up to 5 FPs/image, with the largest improvement observed in the 0.5–2 FPs range.
• When more false positives are allowed, the sensitivity of ULDor with FP retraining reaches 94.2%, surpassing the 88.0% sensitivity reported in [4].
• Despite improvements, approximately 5.8% of true lesions remain undetected, indicating room for further enhancement in handling low-contrast lesions.