[Paper Review] Automatic Liver Segmentation Using an Adversarial Image-to-Image Network

Dong Yang, Daguang Xu+6 more|arXiv (Cornell University)|Jul 25, 2017

Radiomics and Machine Learning in Medical Imaging17 citations

TL;DR

This paper proposes a deep adversarial image-to-image network (DI2IN-AN) for automatic 3D liver segmentation in CT scans, combining a U-Net-like encoder-decoder architecture with multi-level skip connections, deep supervision, and adversarial training to improve boundary accuracy. Trained on 1,000+ diverse CT volumes, the method achieves state-of-the-art performance with a mean symmetric surface distance (ASD) of 1.90 mm and Dice coefficient of 0.95, while running in under one second per scan.

ABSTRACT

Automatic liver segmentation in 3D medical images is essential in many clinical applications, such as pathological diagnosis of hepatic diseases, surgical planning, and postoperative assessment. However, it is still a very challenging task due to the complex background, fuzzy boundary, and various appearance of liver. In this paper, we propose an automatic and efficient algorithm to segment liver from 3D CT volumes. A deep image-to-image network (DI2IN) is first deployed to generate the liver segmentation, employing a convolutional encoder-decoder architecture combined with multi-level feature concatenation and deep supervision. Then an adversarial network is utilized during training process to discriminate the output of DI2IN from ground truth, which further boosts the performance of DI2IN. The proposed method is trained on an annotated dataset of 1000 CT volumes with various different scanning protocols (e.g., contrast and non-contrast, various resolution and position) and large variations in populations (e.g., ages and pathology). Our approach outperforms the state-of-the-art solutions in terms of segmentation accuracy and computing efficiency.

Motivation & Objective

• To address the challenge of accurate and efficient automatic liver segmentation in 3D CT volumes with complex backgrounds, fuzzy boundaries, and high anatomical variability.
• To overcome limitations of traditional post-processing methods like CRF and graph-cut, which suffer from boundary leakage in low-contrast regions.
• To improve deep learning-based segmentation by integrating adversarial training into an image-to-image network, enhancing structural detail and boundary precision.
• To leverage a large-scale dataset of over 1,000 diverse 3D CT volumes to boost generalization and robustness of the model.
• To achieve high segmentation accuracy with minimal inference time, enabling clinical deployment.

Proposed method

• The method employs a deep image-to-image network (DI2IN) with a 3D encoder-decoder architecture, using 3D convolutional and bilinear upsampling layers to preserve spatial resolution.
• Skip connections are used to concatenate features from encoder and decoder layers at multiple levels, enhancing feature reuse and gradient flow.
• Deep supervision is applied at multiple decoder stages, with individual loss functions to guide intermediate predictions and improve training stability.
• An adversarial training component is introduced, where a discriminator network is trained to distinguish between ground truth liver masks and predictions from the generator (DI2IN), encouraging sharper and more realistic segmentation outputs.
• The generator loss combines standard cross-entropy loss with an adversarial loss term, where the discriminator's feedback helps refine the generator’s output distribution.
• The network is trained end-to-end using stochastic gradient descent with a learning rate of 0.01, decayed by 10 after 100 iterations, and adversarial training is performed with a balancing weight λ = 0.01.

Experimental results

Research questions

• RQ1 Can adversarial training improve the quality of liver segmentation in 3D CT volumes, particularly in regions with fuzzy boundaries?
• RQ2 Does training on a large-scale dataset of over 1,000 diverse 3D CT volumes significantly enhance the robustness and accuracy of deep learning-based liver segmentation?
• RQ3 Can an end-to-end image-to-image network with skip connections and deep supervision outperform existing methods without relying on post-processing refinements like CRF or graph-cut?
• RQ4 How does the proposed DI2IN-AN framework compare in performance and inference speed to state-of-the-art methods on unseen clinical data?
• RQ5 To what extent does adversarial training reduce boundary leakage in low-contrast liver regions compared to conventional refinement techniques?

Key findings

• The proposed DI2IN-AN achieved a mean symmetric surface distance (ASD) of 1.90 mm and a Dice coefficient of 0.95 on an independent test set of 50 cases, outperforming all baseline methods.
• Training with 1,000+ CT volumes reduced the mean ASD by 0.23 mm and the maximum ASD by 3.84 mm compared to training with only 400 volumes.
• The adversarial component reduced the maximum ASD error from 7.82 mm (DI2IN-AN with 400 cases) to 6.32 mm (DI2IN-AN with 1,000 cases), indicating improved boundary accuracy.
• The method achieved inference time of less than one second per 3D volume on an NVIDIA TITAN X GPU, significantly faster than prior methods such as the 1.5-minute inference time reported in [6].
• CRF and graph-cut post-processing methods worsened segmentation performance on low-contrast data, confirming the advantage of end-to-end adversarial learning in handling fuzzy boundaries.
• The DI2IN-AN model trained on 1,000+ cases achieved a 20% improvement in average symmetric surface distance over baseline networks, demonstrating the benefit of large-scale data.