[Paper Review] T-Net: Encoder-Decoder in Encoder-Decoder architecture for the main vessel segmentation in coronary angiography.

Tae Joon Jun, Jihoon Kweon+2 more|arXiv (Cornell University)|May 10, 2019

Retinal Imaging and AnalysisMedicine21 references3 citations

TL;DR

T-Net proposes an encoder-decoder-in-encoder-decoder (EDiED) architecture that enhances vessel segmentation in coronary angiography by enabling multiple skip connections across feature map sizes. By strategically arranging pooling and up-sampling within a single block, T-Net delivers low- to high-level features directly to the decoder’s start, achieving a Dice Similarity Coefficient of 0.890—0.170 higher than U-Net.

ABSTRACT

In this paper, we proposed T-Net containing a small encoder-decoder inside the encoder-decoder structure (EDiED). T-Net overcomes the limitation that U-Net can only have a single set of the concatenate layer between encoder and decoder block. To be more precise, the U-Net symmetrically forms the concatenate layers, so the low-level feature of the encoder is connected to the latter part of the decoder, and the high-level feature is connected to the beginning of the decoder. T-Net arranges the pooling and up-sampling appropriately during the encoder process, and likewise during the decoding process so that feature-maps of various sizes are obtained in a single block. As a result, all features from the low-level to the high-level extracted from the encoder are delivered from the beginning of the decoder to predict a more accurate mask. We evaluated T-Net for the problem of segmenting three main vessels in coronary angiography images. The experiment consisted of a comparison of U-Net and T-Nets under the same conditions, and an optimized T-Net for the main vessel segmentation. As a result, T-Net recorded a Dice Similarity Coefficient score (DSC) of 0.815, 0.095 higher than that of U-Net, and the optimized T-Net recorded a DSC of 0.890 which was 0.170 higher than that of U-Net. In addition, we visualized the weight activation of the convolutional layer of T-Net and U-Net to show that T-Net actually predicts the mask from earlier decoders. Therefore, we expect that T-Net can be effectively applied to other similar medical image segmentation problems.

Motivation & Objective

• Address the limitation of U-Net’s single skip connection, which restricts feature fusion between encoder and decoder.
• Enable more effective feature propagation by delivering all encoder features—low-level to high-level—early in the decoder.
• Improve accuracy in segmenting the three main coronary vessels in angiographic images.
• Design a scalable architecture applicable to other medical image segmentation tasks with similar hierarchical feature needs.

Proposed method

• Introduce an EDiED (encoder-decoder in encoder-decoder) structure where a small encoder-decoder is embedded within the main encoder-decoder.
• Reorganize pooling and up-sampling operations during encoding and decoding to generate feature maps of varying sizes within a single block.
• Establish multiple skip connections from all encoder feature maps to the beginning of the decoder, enabling early and comprehensive feature fusion.
• Utilize symmetric encoding and decoding paths with adaptive feature map size management to preserve spatial and semantic information.
• Train the network end-to-end using binary cross-entropy loss for pixel-wise segmentation of coronary vessels.
• Apply visualization techniques to analyze convolutional layer activations and validate the model’s attention to early decoder predictions.

Experimental results

Research questions

• RQ1 Can an enhanced encoder-decoder architecture with multiple skip connections improve vessel segmentation accuracy in coronary angiography?
• RQ2 Does delivering low- and high-level features simultaneously to the decoder’s start lead to better mask prediction than U-Net’s single skip connection?
• RQ3 How does the proposed T-Net architecture compare to U-Net in terms of Dice Similarity Coefficient for main coronary vessel segmentation?
• RQ4 To what extent does the model leverage early decoder layers for accurate segmentation, as indicated by activation patterns?

Key findings

• T-Net achieved a Dice Similarity Coefficient (DSC) of 0.815 on the main vessel segmentation task, which is 0.095 higher than U-Net’s performance.
• An optimized version of T-Net achieved a DSC of 0.890, representing a 0.170 improvement over U-Net.
• Visualization of convolutional layer weights confirmed that T-Net utilizes early decoder layers for mask prediction, indicating effective feature utilization.
• The multi-scale skip connection mechanism in T-Net enables more robust feature fusion across different levels of abstraction.
• The architecture demonstrates superior generalization for medical image segmentation tasks requiring precise boundary detection.
• T-Net’s design is transferable to other medical imaging applications with similar hierarchical feature requirements.