[Paper Review] Context-aware PolyUNet for Liver and Lesion Segmentation from Abdominal CT Images

Liping Zhang, Simon C.H. Yu|arXiv (Cornell University)|Jun 21, 2021

Advanced Neural Network Applications17 references4 citations

TL;DR

This paper proposes a context-aware PolyUNet for accurate liver and liver lesion segmentation in abdominal CT scans, leveraging multiple paths in a polynomial fashion and consecutive t-adjacent slices to enhance spatial context while reducing GPU memory usage. Using a zoom-out/in and two-stage refinement strategy, it achieves state-of-the-art performance, ranking 3rd in liver segmentation, 2nd in lesion detection, 12th in lesion segmentation, and 5th in tumor burden estimation at the MICCAI 2017 LiTS challenge with a single model.

ABSTRACT

Accurate liver and lesion segmentation from computed tomography (CT) images are highly demanded in clinical practice for assisting the diagnosis and assessment of hepatic tumor disease. However, automatic liver and lesion segmentation from contrast-enhanced CT volumes is extremely challenging due to the diversity in contrast, resolution, and quality of images. Previous methods based on UNet for 2D slice-by-slice or 3D volume-by-volume segmentation either lack sufficient spatial contexts or suffer from high GPU computational cost, which limits the performance. To tackle these issues, we propose a novel context-aware PolyUNet for accurate liver and lesion segmentation. It jointly explores structural diversity and consecutive t-adjacent slices to enrich feature expressive power and spatial contextual information while avoiding the overload of GPU memory consumption. In addition, we utilize zoom out/in and two-stage refinement strategy to exclude the irrelevant contexts and focus on the specific region for the fine-grained segmentation. Our method achieved very competitive performance at the MICCAI 2017 Liver Tumor Segmentation (LiTS) Challenge among all tasks with a single model and ranked the $3^{rd}$, $12^{th}$, $2^{nd}$, and $5^{th}$ places in the liver segmentation, lesion segmentation, lesion detection, and tumor burden estimation, respectively.

Motivation & Objective

• To address the challenge of accurate automatic liver and lesion segmentation in contrast-enhanced abdominal CT scans, which are complicated by low tissue contrast, artifacts, and imaging variability.
• To overcome limitations of existing 2D and 3D UNet-based methods that either lack sufficient spatial context or incur high computational costs and GPU memory usage.
• To improve feature representation and segmentation accuracy by integrating structural diversity through multiple paths in a polynomial combination and leveraging temporal-spatial context from consecutive slices.
• To refine segmentation results by excluding irrelevant surrounding contexts and focusing on target regions through a zoom-out/in mechanism and two-stage refinement strategy.

Proposed method

• The proposed context-aware PolyUNet integrates multiple parallel paths in a polynomial combination to enhance feature expressive power beyond depth and width.
• It employs UNet-style long-range skip connections to combine low-level and high-level features, preserving spatial resolution and enabling multi-scale context learning.
• The network leverages t-adjacent slices (e.g., t-1, t, t+1) to jointly capture intra- and inter-slice spatial features, effectively modeling 3D context in a 2D framework to reduce computational load.
• A zoom-out/in strategy is applied to expand the receptive field and then refine the region of interest, helping to exclude irrelevant background contexts.
• A two-stage refinement strategy is used to progressively focus on the lesion region, improving localization accuracy and fine-grained segmentation.
• The model is trained end-to-end using a single network for joint liver and lesion segmentation, minimizing inference overhead and memory consumption.

Experimental results

Research questions

• RQ1 Can a 2D-based network effectively capture 3D spatial context in abdominal CT volumes without the high computational cost of 3D networks?
• RQ2 How can structural diversity in network architecture improve feature representation for liver and lesion segmentation across varying lesion sizes and shapes?
• RQ3 To what extent can consecutive t-adjacent slices enhance segmentation performance compared to single-slice processing?
• RQ4 Can a zoom-out/in and two-stage refinement strategy effectively suppress background noise and improve fine-grained segmentation of small or low-contrast lesions?
• RQ5 Does a single-model approach achieve competitive performance across multiple tasks (liver segmentation, lesion segmentation, detection, and tumor burden estimation) in a multi-task medical image segmentation challenge?

Key findings

• The method achieved a Dice score of 0.9600 for liver segmentation, ranking 3rd out of all teams in the LiTS-MICCAI 2017 challenge.
• For lesion segmentation, the method achieved a Dice score of 0.6200, ranking 12th among all submissions.
• In lesion detection, the method achieved a recall of 0.446, securing the 2nd place in the challenge.
• For tumor burden estimation, the method achieved an RMSE of 0.0220, ranking 5th among all teams.
• The model achieved competitive performance across all four tasks using a single architecture, demonstrating robustness and generalization across diverse segmentation challenges.