[Paper Review] GAN based Unsupervised Segmentation: Should We Match the Exact Number of Objects

Quan Liu, Isabella Gaeta+3 more|arXiv (Cornell University)|Oct 22, 2020

Cell Image Analysis Techniques20 references4 citations

TL;DR

This paper proposes a novel GAN-based unsupervised segmentation method for highly overlapping sub-cellular microvilli using fluorescence microscopy. It introduces micro-level matching to align object counts per mini-batch, but finds that macro-level matching—especially with Gaussian smoothing and noise augmentation—yields superior segmentation performance, outperforming micro-level matching in both qualitative and quantitative results (Dice score: 0.6169 vs. 0.5730).

ABSTRACT

The unsupervised segmentation is an increasingly popular topic in biomedical image analysis. The basic idea is to approach the supervised segmentation task as an unsupervised synthesis problem, where the intensity images can be transferred to the annotation domain using cycle-consistent adversarial learning. The previous studies have shown that the macro-level (global distribution level) matching on the number of the objects (e.g., cells, tissues, protrusions etc.) between two domains resulted in better segmentation performance. However, no prior studies have exploited whether the unsupervised segmentation performance would be further improved when matching the exact number of objects at micro-level (mini-batch level). In this paper, we propose a deep learning based unsupervised segmentation method for segmenting highly overlapped and dynamic sub-cellular microvilli. With this challenging task, both micro-level and macro-level matching strategies were evaluated. To match the number of objects at the micro-level, the novel fluorescence-based micro-level matching approach was presented. From the experimental results, the micro-level matching did not improve the segmentation performance, compared with the simpler macro-level matching.

Motivation & Objective

• To develop the first deep learning-based unsupervised segmentation method for sub-cellular microvilli in fluorescence microscopy.
• To investigate whether micro-level (mini-batch level) object count matching improves segmentation performance over standard macro-level (global distribution) matching.
• To evaluate the impact of data augmentation strategies—smoothing, noise, and brightness—on simulated mask generation for unsupervised training.
• To enable object counting without extra human annotation by leveraging the multi-channel nature of fluorescence images.

Proposed method

• Utilizes CycleGAN for cycle-consistent image synthesis between real intensity images and simulated binary masks of microvilli.
• Employs the green fluorescence channel to automatically count microvilli in real images for micro-level matching within each mini-batch.
• Generates simulated masks with matching object counts per mini-batch using a dedicated simulator, enabling micro-level pairing.
• Applies data augmentation (Gaussian smoothing, random noise, brightness variation) to simulated masks to improve generalization.
• Trains a U-Net segmentation network on synthetic image-mask pairs without any human-annotated data, using only CycleGAN-generated data.
• Evaluates segmentation performance using Dice similarity coefficient at multiple ground truth widths (w=1 to w=5), with final predictions reconstructed from 128×128 patches.

Experimental results

Research questions

• RQ1 Does micro-level matching of object counts per mini-batch improve unsupervised segmentation performance compared to macro-level matching?
• RQ2 Can the multi-channel nature of fluorescence microscopy enable accurate, automated object counting for micro-level pairing without additional annotation?
• RQ3 How do different data augmentation strategies (smoothing, noise, brightness) affect the quality of synthesized masks and downstream segmentation?
• RQ4 Is macro-level matching more robust than micro-level matching when generalizing to unseen test data?
• RQ5 What is the optimal combination of augmentation and matching strategy for unsupervised microvilli segmentation?

Key findings

• Macro-level matching with Gaussian smoothing and random noise augmentation achieved the highest Dice score of 0.6169 at width w=5, outperforming all micro-level configurations.
• Micro-level matching did not improve segmentation performance; the best micro-level result (0.5730 Dice) was lower than the best macro-level result.
• The U-Net trained on macro-level paired data generalized better to new test data than the model trained on micro-level paired data.
• Micro-level pairing achieved higher training-set accuracy due to finer alignment, but this did not translate to better generalization.
• The standard macro-level matching strategy remains more robust and effective than micro-level matching for unsupervised sub-cellular segmentation.
• The combination of Gaussian smoothing and random noise on simulated masks significantly enhanced segmentation performance across all evaluation widths.