[Paper Review] Self-supervised Tumor Segmentation through Layer Decomposition

Xiaoman Zhang, Weidi Xie+5 more|arXiv (Cornell University)|Sep 7, 2021

AI in cancer detection71 references4 citations

TL;DR

This paper proposes a self-supervised tumor segmentation method using layer decomposition on synthetic tumor data, enabling zero-shot inference on real medical images without fine-tuning. By training a model to decompose synthetic 3D volumes into normal tissue and tumor layers, it achieves state-of-the-art zero-shot performance on BraTS2018 and LiTS2017, with superior generalization and transferability under low-annotation regimes.

ABSTRACT

In this paper, we target self-supervised representation learning for zero-shot tumor segmentation. We make the following contributions: First, we advocate a zero-shot setting, where models from pre-training should be directly applicable for the downstream task, without using any manual annotations. Second, we take inspiration from "layer-decomposition", and innovate on the training regime with simulated tumor data. Third, we conduct extensive ablation studies to analyse the critical components in data simulation, and validate the necessity of different proxy tasks. We demonstrate that, with sufficient texture randomization in simulation, model trained on synthetic data can effortlessly generalise to segment real tumor data. Forth, our approach achieves superior results for zero-shot tumor segmentation on different downstream datasets, BraTS2018 for brain tumor segmentation and LiTS2017 for liver tumor segmentation. While evaluating the model transferability for tumor segmentation under a low-annotation regime, the proposed approach also outperforms all existing self-supervised approaches, opening up the usage of self-supervised learning in practical scenarios.

Motivation & Objective

• Address the challenge of high annotation cost in medical image segmentation by enabling zero-shot tumor segmentation without manual annotations.
• Reduce the domain gap between synthetic training data and real clinical scans through realistic texture randomization in data simulation.
• Develop a self-supervised training regime based on layer decomposition that directly predicts tumor masks without fine-tuning.
• Demonstrate strong transferability of the pre-trained model to real tumor segmentation tasks, even with minimal or no labeled data.
• Validate the necessity and effectiveness of proxy tasks and data simulation components through extensive ablation studies.

Proposed method

• Generate synthetic 3D medical volumes by blending artificial tumors into normal organ structures using alpha blending, simulating realistic tumor appearance.
• Train a deep neural network to reconstruct the normal organ, the tumor, and the tumor mask from the synthetic input, using a layer-decomposition objective.
• Formulate the self-supervised proxy task as a reconstruction problem where the model learns to separate the input into tissue and tumor components.
• Use texture randomization during data generation to improve generalization to real tumor appearances and reduce domain shift.
• Apply the pre-trained model directly to real scans for inference, predicting tumor masks without any fine-tuning (zero-shot setting).
• Enable efficient fine-tuning when a few annotations are available, demonstrating strong transferability in low-data regimes.

Experimental results

Research questions

• RQ1 Can a self-supervised model trained on synthetic tumor data generalize directly to real tumor segmentation without fine-tuning?
• RQ2 How does texture randomization in synthetic data generation affect the model’s ability to generalize to real tumors?
• RQ3 What is the impact of different proxy tasks (e.g., reconstruction, decomposition) on downstream tumor segmentation performance?
• RQ4 How does the proposed method compare to existing self-supervised approaches in terms of zero-shot and low-shot transferability?
• RQ5 What components in the data simulation pipeline are most critical for achieving strong generalization to real clinical data?

Key findings

• The proposed method achieves a Dice score of 90.10 ± 0.51 on BraTS2018 for whole tumor segmentation in the zero-shot setting, outperforming all existing self-supervised methods.
• On LiTS2017, the model achieves a Dice score of 81.39 ± 0.64 for whole tumor segmentation under zero-shot inference, demonstrating strong generalization.
• With only 10% of training data and full fine-tuning, the model achieves a Dice score of 99.86 ± 0.01 on BraTS2018 enhanced tumor segmentation, surpassing SOTA.
• Ablation studies confirm that sufficient texture randomization in synthetic data is essential for generalization, reducing domain shift and improving robustness.
• The model shows lower standard deviation across runs compared to baselines, indicating higher training stability and robustness in the self-supervised regime.
• Even without fine-tuning, the model outperforms supervised training from scratch without data augmentation, highlighting the effectiveness of the self-supervised pre-training.