[Paper Review] Learning More with Less: Conditional PGGAN-based Data Augmentation for Brain Metastases Detection Using Highly-Rough Annotation on MR Images

Changhee Han, Kohei Murao+6 more|arXiv (Cornell University)|Feb 26, 2019

AI in cancer detection37 references80 citations

TL;DR

This paper proposes Conditional Progressive Growing GANs (CPGGANs), a novel data augmentation method that generates realistic, diverse 256×256 brain MR images with tumors placed at desired positions and sizes using only highly-rough bounding box annotations. By incrementally conditioning PGGANs on these coarse labels, the approach boosts tumor detection sensitivity by 10% with clinically acceptable false positives, even when trained without additional normal brain images.

ABSTRACT

Accurate Computer-Assisted Diagnosis, associated with proper data wrangling, can alleviate the risk of overlooking the diagnosis in a clinical environment. Towards this, as a Data Augmentation (DA) technique, Generative Adversarial Networks (GANs) can synthesize additional training data to handle the small/fragmented medical imaging datasets collected from various scanners; those images are realistic but completely different from the original ones, filling the data lack in the real image distribution. However, we cannot easily use them to locate disease areas, considering expert physicians' expensive annotation cost. Therefore, this paper proposes Conditional Progressive Growing of GANs (CPGGANs), incorporating highly-rough bounding box conditions incrementally into PGGANs to place brain metastases at desired positions/sizes on 256 X 256 Magnetic Resonance (MR) images, for Convolutional Neural Network-based tumor detection; this first GAN-based medical DA using automatic bounding box annotation improves the training robustness. The results show that CPGGAN-based DA can boost 10% sensitivity in diagnosis with clinically acceptable additional False Positives. Surprisingly, further tumor realism, achieved with additional normal brain MR images for CPGGAN training, does not contribute to detection performance, while even three physicians cannot accurately distinguish them from the real ones in Visual Turing Test.

Motivation & Objective

• To address the challenge of limited and fragmented medical imaging datasets with sparse, inconsistent annotations in brain metastases detection.
• To develop a data augmentation method that generates realistic, diverse pathological MR images without requiring precise segmentation or benign image inputs.
• To enable high-sensitivity tumor detection using only coarse bounding box annotations, minimizing expert annotation burden.
• To evaluate whether enhanced tumor realism in synthetic images improves downstream detection performance.

Proposed method

• CPGGANs extend Progressive Growing GANs (PGGANs) by incrementally injecting highly-rough bounding box conditions into the generator during training to control tumor position and size.
• The method conditions the GAN on bounding box masks rather than full segmentation maps, enabling generation of whole-brain MR images with tumors at desired locations.
• Conditional noise injection is used to introduce variability in tumor shape and appearance while maintaining anatomical plausibility.
• The generator is trained to produce 256×256 MR images with tumors that are visually indistinguishable from real ones, as validated by a Visual Turing Test with expert radiologists.
• T-SNE visualization is used to compare the distribution of real and synthetic tumor patches, assessing data diversity and distributional alignment.
• YOLOv3 is used as the object detector, trained on a combination of real and synthetic images to evaluate detection performance.

Experimental results

Research questions

• RQ1 How can PGGANs be modified to naturally place tumors of random shape at desired positions/sizes using only highly-rough bounding box masks?
• RQ2 Can GAN-based data augmentation with coarse annotations improve detection sensitivity without excessive false positives?
• RQ3 Does increased realism in generated tumors, as judged by experts, lead to better detection performance?
• RQ4 How does the inclusion of additional normal brain MR images during CPGGAN training affect realism and detection performance?

Key findings

• CPGGAN-based data augmentation increases tumor detection sensitivity by 10% compared to baseline models trained only on real data.
• The method achieves this improvement with a clinically acceptable increase in false positives, demonstrating practical utility in clinical settings.
• Despite higher realism in tumor appearance when trained with additional normal brain images, detection performance did not improve further, indicating that realism alone does not enhance detection.
• Three expert physicians could not reliably distinguish CPGGAN-generated tumor bounding boxes from real ones in a Visual Turing Test when trained with normal brain images, confirming high realism.
• T-SNE visualization showed that synthetic tumor patches have a moderately similar and complementary distribution to real ones, supporting effective data augmentation.
• Over-reliance on synthetic data led to performance degradation, suggesting an optimal balance between real and synthetic data is critical for robust training.