[Paper Review] Semantic Segmentation of Colon Glands with Deep Convolutional Neural Networks and Total Variation Segmentation

Philipp Kainz, Michael Pfeiffer+1 more|arXiv (Cornell University)|Nov 21, 2015

Radiomics and Machine Learning in Medical Imaging21 citations

TL;DR

This paper presents a deep learning framework for semantic segmentation of colon glands in H&E-stained histopathology slides, combining two convolutional neural networks (Object-Net and Separator-Net) with total variation-based figure-ground segmentation to improve accuracy. The method achieves 98.33% and 93.75% tissue classification accuracy on test sets A and B, respectively, while enabling precise distinction between benign and malignant tissue.

ABSTRACT

Segmentation of histopathology sections is an ubiquitous requirement in digital pathology and due to the large variability of biological tissue, machine learning techniques have shown superior performance over standard image processing methods. As part of the GlaS@MICCAI2015 colon gland segmentation challenge, we present a learning-based algorithm to segment glands in tissue of benign and malignant colorectal cancer. Images are preprocessed according to the Hematoxylin-Eosin staining protocol and two deep convolutional neural networks (CNN) are trained as pixel classifiers. The CNN predictions are then regularized using a figure-ground segmentation based on weighted total variation to produce the final segmentation result. On two test sets, our approach achieves a tissue classification accuracy of 98% and 94%, making use of the inherent capability of our system to distinguish between benign and malignant tissue.

Motivation & Objective

• To address the high variability in glandular structures across benign and malignant colorectal cancer tissues in histopathology images.
• To improve semantic segmentation performance by combining deep learning with a globally optimal regularization technique.
• To enable accurate discrimination between benign and malignant tissue using a multi-class classification approach within a deep learning framework.
• To refine gland segmentation by learning and applying a dedicated separator network that distinguishes touching or overlapping glands.

Proposed method

• Preprocessing involves color deconvolution of H&E-stained images to isolate the hematoxylin component and enhance tissue structure, followed by contrast-limited adaptive histogram equalization (CLAHE) for illumination robustness.
• Two deep convolutional neural networks are trained: Object-Net for classifying pixels as gland or background, and Separator-Net for identifying gland-separating structures to resolve touching glands.
• Classifier outputs are combined and regularized using a convex geodesic active contour formulation based on weighted total variation, minimizing contour length to produce globally optimal figure-ground segmentation.
• The method uses a four-class pixel classification setup, where Object-Net outputs probabilities for background, benign glands, malignant glands, and separator structures.
• Final segmentation is derived from the regularized classifier outputs, with shape and boundary fidelity improved through the total variation minimization process.
• Tissue classification is performed by aggregating per-pixel probabilities across the image domain, with the final label assigned based on the maximum average probability for benign or malignant tissue.

Experimental results

Research questions

• RQ1 Can a deep learning-based approach with dual CNNs improve semantic segmentation of colon glands in H&E-stained histopathology images compared to standard image processing?
• RQ2 How effective is the integration of a dedicated separator network in resolving overlapping or touching glands?
• RQ3 To what extent does total variation regularization enhance segmentation accuracy and boundary precision in gland segmentation?
• RQ4 Can the model simultaneously achieve high accuracy in both gland segmentation and benign/malignant tissue classification?

Key findings

• The proposed method achieved a tissue classification accuracy of 98.33% on test set A and 93.75% on test set B, demonstrating strong performance in distinguishing benign and malignant tissue.
• The addition of the Separator-Net significantly improved segmentation metrics, including F1-score, Dice coefficient, and Hausdorff distance, due to better handling of overlapping glands.
• The average decision confidence for benign cases was 0.84 (SD 0.13) on test set A and 0.74 (SD 0.11) on test set B, indicating reliable model confidence.
• The average decision confidence for malignant cases was 0.81 (SD 0.11) on test set A and 0.86 (SD 0.15) on test set B, showing consistent performance across test sets.
• Qualitative results showed that the method effectively segmented glands even in complex regions with overlapping or irregularly shaped glands, with only minor false positives and negatives in challenging cases.
• The total variation regularization step successfully produced globally optimal segmentations by minimizing contour length while preserving object boundaries, enhancing shape fidelity.