[Paper Review] Learning Normalized Inputs for Iterative Estimation in Medical Image Segmentation

Michal Drozdzal, Gabriel Chartrand+7 more|arXiv (Cornell University)|Feb 16, 2017

Advanced Neural Network Applications38 references5 citations

TL;DR

This paper proposes a novel medical image segmentation pipeline that uses a lightweight Fully Convolutional Network (FCN) as a trainable pre-processor to normalize input images before feeding them into a deep Fully Convolutional Residual Network (FC-ResNet). The method achieves state-of-the-art performance on the EM ISBI 2012 benchmark, improves liver lesion segmentation over standard FCNs, and delivers competitive results on 3D MRI prostate segmentation, demonstrating strong versatility across modalities and anatomical regions.

ABSTRACT

In this paper, we introduce a simple, yet powerful pipeline for medical image segmentation that combines Fully Convolutional Networks (FCNs) with Fully Convolutional Residual Networks (FC-ResNets). We propose and examine a design that takes particular advantage of recent advances in the understanding of both Convolutional Neural Networks as well as ResNets. Our approach focuses upon the importance of a trainable pre-processing when using FC-ResNets and we show that a low-capacity FCN model can serve as a pre-processor to normalize medical input data. In our image segmentation pipeline, we use FCNs to obtain normalized images, which are then iteratively refined by means of a FC-ResNet to generate a segmentation prediction. As in other fully convolutional approaches, our pipeline can be used off-the-shelf on different image modalities. We show that using this pipeline, we exhibit state-of-the-art performance on the challenging Electron Microscopy benchmark, when compared to other 2D methods. We improve segmentation results on CT images of liver lesions, when contrasting with standard FCN methods. Moreover, when applying our 2D pipeline on a challenging 3D MRI prostate segmentation challenge we reach results that are competitive even when compared to 3D methods. The obtained results illustrate the strong potential and versatility of the pipeline by achieving highly accurate results on multi-modality images from different anatomical regions and organs.

Motivation & Objective

• To address the sensitivity of deep FC-ResNets to raw medical image input distributions, which can hinder performance.
• To improve segmentation accuracy across diverse medical imaging modalities, including electron microscopy, CT, and MRI.
• To explore the role of pre-processing in enabling effective iterative refinement in FC-ResNets, leveraging their residual architecture.
• To demonstrate that a low-capacity FCN can serve as an effective, learnable pre-processor, outperforming traditional hand-crafted normalization methods.
• To validate the pipeline’s generalization and robustness across 2D and 3D medical imaging tasks.

Proposed method

• A lightweight FCN is trained end-to-end as a pre-processor to normalize raw medical image inputs, learning modality-specific intensity distributions.
• The output of the pre-processor is fed into a very deep FC-ResNet (over 100 layers) that performs iterative refinement of the normalized input through residual blocks.
• The FC-ResNet uses skip connections and batch normalization to enable deep optimization and stable gradient flow, enabling iterative estimation of the final segmentation map.
• The pipeline is fully convolutional, allowing direct application to different image modalities without architectural changes.
• The pre-processor is trained jointly with the FC-ResNet in an end-to-end fashion, enabling the network to learn optimal input normalization for segmentation.
• The method leverages the iterative inference interpretation of ResNets, where the FCN provides an initial proposal and the FC-ResNet refines it through multiple residual updates.

Experimental results

Research questions

• RQ1 Can a learnable, lightweight FCN pre-processor significantly improve the performance of deep FC-ResNets in medical image segmentation?
• RQ2 How does the proposed pipeline compare to standard hand-crafted pre-processing techniques in terms of segmentation accuracy across diverse imaging modalities?
• RQ3 To what extent can a 2D FC-ResNet pipeline achieve competitive results on challenging 3D medical image segmentation tasks?
• RQ4 Does the integration of a pre-processor align with the iterative estimation interpretation of ResNets, and does it enhance model robustness?
• RQ5 Can the proposed framework generalize across different anatomical regions and imaging protocols without architectural modification?

Key findings

• The proposed pipeline achieves state-of-the-art performance on the challenging EM ISBI 2012 benchmark, outperforming other 2D methods.
• On a 3D MRI prostate segmentation task, the 2D pipeline achieved competitive results, demonstrating strong generalization even when applied to 3D data.
• The method improved segmentation accuracy on liver lesion CT images compared to standard FCN baselines, highlighting the benefit of learnable pre-processing.
• The FCN pre-processor effectively normalizes input data, reducing inter-scan intensity variability and improving downstream segmentation performance.
• The pipeline’s performance is significantly enhanced when using a trainable pre-processor, indicating that FC-ResNets are highly sensitive to input distribution and benefit from such normalization.
• The results support the interpretation of FC-ResNets as performing iterative estimation, where the pre-processor provides a refined initial input for progressive refinement.