[Paper Review] Domain Adaptation-based Augmentation for Weakly Supervised Nuclei Detection

Nicolas Brieu, Armin Meier+5 more|arXiv (Cornell University)|Jul 10, 2019

AI in cancer detection17 references13 citations

TL;DR

This paper proposes a weakly supervised, domain adaptation-based augmentation method for nuclei detection in histopathology images, using unpaired image-to-image translation (CycleGAN) to generate synthetic labeled images from a source domain (e.g., immunofluorescence) to a target domain (e.g., H&E), enabling training of a detection network without target-domain annotations. The approach achieves f1 scores of 0.85 on unseen H&E datasets—comparable to fully supervised methods—demonstrating state-of-the-art performance under annotation scarcity.

ABSTRACT

The detection of nuclei is one of the most fundamental components of computational pathology. Current state-of-the-art methods are based on deep learning, with the prerequisite that extensive labeled datasets are available. The increasing number of patient cohorts to be analyzed, the diversity of tissue stains and indications, as well as the cost of dataset labeling motivates the development of novel methods to reduce labeling effort across domains. We introduce in this work a weakly supervised 'inter-domain' approach that (i) performs stain normalization and unpaired image-to-image translation to transform labeled images on a source domain to synthetic labeled images on an unlabeled target domain and (ii) uses the resulting synthetic labeled images to train a detection network on the target domain. Extensive experiments show the superiority of the proposed approach against the state-of-the-art 'intra-domain' detection based on fully-supervised learning.

Motivation & Objective

• Address the high cost and scarcity of annotated histopathology datasets across diverse tissue stains and patient cohorts.
• Reduce labeling effort in digital pathology by enabling nuclei detection in a target domain using only annotations from a different source domain.
• Overcome limitations of standard two-step domain adaptation by generating multiple diverse synthetic images per source image to improve detection robustness.
• Develop a weakly supervised method that leverages unpaired, unsupervised domain translation to transfer knowledge across staining modalities without requiring target-domain annotations.

Proposed method

• Apply intra-domain stain normalization (e.g., DCGMM) to align source and target domain images before domain translation.
• Use CycleGAN for unpaired, one-to-one image-to-image translation to synthesize realistic target-domain images from source-domain labeled images.
• Generate multiple synthetic images per source image to form an augmented training set, enhancing diversity and generalization.
• Train a nuclei detection network (e.g., CNN) on the ensemble of synthetic labeled images, using only source-domain annotations.
• Introduce a cross-domain supervision strategy that combines synthetic source images with a small number of real target-domain annotations to further improve performance.
• Select the best-performing CycleGAN model from multiple iterations based on visual quality and detection accuracy, avoiding suboptimal single-model fine-tuning.

Experimental results

Research questions

• RQ1 Can unpaired domain adaptation via CycleGAN effectively transfer nuclei detection knowledge from a labeled source domain to an unlabeled target domain with different staining protocols?
• RQ2 Does generating multiple diverse synthetic images per source image improve detection performance compared to using a single synthetic image?
• RQ3 How does the performance of weakly supervised inter-domain detection compare to fully supervised intra-domain detection when target-domain annotations are scarce?
• RQ4 Can cross-domain supervision—combining synthetic source images with limited real target images—outperform both pure inter-domain and intra-domain approaches?
• RQ5 Is the performance gain from ensemble synthetic data robust across different histopathology datasets and staining types?

Key findings

• The proposed inter-domain method achieved an f1 score of 0.85 on the unseen NSCLC IF dataset when using only synthetic labels from the HE source domain, matching the performance of fully supervised training on real IF labels.
• In the more challenging IF2HE direction, the method achieved f1 scores of 0.85 and 0.81 on the MoNuSeg and TNBC HE test datasets, respectively—comparable to fully supervised methods (f1 = 0.86 and 0.84).
• When target-domain annotations were limited (N_HE ≤ 18K), weak inter-domain supervision outperformed full intra-domain supervision, demonstrating its value under annotation scarcity.
• Cross-domain supervision, combining synthetic and real target images, outperformed both pure inter-domain and intra-domain methods when target annotations were limited, and matched intra-domain performance with more data.
• Using only the single best CycleGAN model for synthetic image generation led to a significant drop in detection accuracy, highlighting the importance of model ensemble and diversity in synthetic data.
• The method’s superiority was statistically validated via paired t-tests, confirming that the performance gains from the proposed approach are significant and consistent across datasets.