[Paper Review] Contour Transformer Network for One-shot Segmentation of Anatomical Structures

Yuhang Lu, Kang Zheng+9 more|arXiv (Cornell University)|Dec 2, 2020

Medical Image Segmentation Techniques50 references4 citations

TL;DR

This paper proposes Contour Transformer Network (CTN), a one-shot medical image segmentation method that models anatomical structure segmentation as a contour evolution process using graph convolutional networks (GCNs). By leveraging one labeled exemplar and unlabeled images through shape, appearance, and edge consistency losses, CTN achieves performance competitive with fully supervised methods and surpasses them with minimal human-in-the-loop feedback via partial contour corrections.

ABSTRACT

Accurate segmentation of anatomical structures is vital for medical image analysis. The state-of-the-art accuracy is typically achieved by supervised learning methods, where gathering the requisite expert-labeled image annotations in a scalable manner remains a main obstacle. Therefore, annotation-efficient methods that permit to produce accurate anatomical structure segmentation are highly desirable. In this work, we present Contour Transformer Network (CTN), a one-shot anatomy segmentation method with a naturally built-in human-in-the-loop mechanism. We formulate anatomy segmentation as a contour evolution process and model the evolution behavior by graph convolutional networks (GCNs). Training the CTN model requires only one labeled image exemplar and leverages additional unlabeled data through newly introduced loss functions that measure the global shape and appearance consistency of contours. On segmentation tasks of four different anatomies, we demonstrate that our one-shot learning method significantly outperforms non-learning-based methods and performs competitively to the state-of-the-art fully supervised deep learning methods. With minimal human-in-the-loop editing feedback, the segmentation performance can be further improved to surpass the fully supervised methods.

Motivation & Objective

• To address the challenge of limited expert-annotated medical image data in training accurate segmentation models.
• To develop a one-shot segmentation method that requires only one labeled exemplar and a set of unlabeled images, eliminating dependency on large external datasets.
• To incorporate an efficient human-in-the-loop mechanism that improves performance through partial contour corrections.
• To exploit the inherent regularity of anatomical structures—such as consistent shape and appearance patterns—across different X-ray images for semi-supervised learning.
• To achieve high-accuracy segmentation in clinical applications where precision is critical, such as joint space measurement.

Proposed method

• Formulates segmentation as a contour evolution process guided by a cascaded graph convolutional network (GCN) to model shape and appearance evolution.
• Introduces three differentiable loss functions: contour perceptual loss (for appearance similarity in VGG feature space), contour bending loss (for shape similarity via thin-plate spline energy), and edge loss (for boundary consistency).
• Uses the exemplar’s contour as initialization and evolves it on unlabeled images using the three loss functions to enforce global shape and appearance consistency.
• Incorporates human-in-the-loop feedback by treating manual corrections as partial contours and applying Chamfer loss to fine-tune the model.
• Employs a semi-supervised training strategy that leverages unlabeled data without requiring external labeled datasets.
• Utilizes a 2D contour-based representation to enable direct comparison of segmentation shapes and boundary features across images.

Experimental results

Research questions

• RQ1 Can a one-shot segmentation model achieve performance comparable to fully supervised methods using only one labeled exemplar and unlabeled data?
• RQ2 Can shape and appearance similarities across anatomical structures in different X-ray images be effectively exploited for semi-supervised learning?
• RQ3 How can a human-in-the-loop mechanism be natively integrated into one-shot segmentation to improve accuracy with minimal annotation effort?
• RQ4 Can contour-based representation and graph convolutional networks effectively model the evolution of anatomical boundaries in medical images?
• RQ5 What is the impact of incorporating partial manual corrections on the performance of one-shot segmentation models?

Key findings

• CTN significantly outperforms non-learning-based one-shot segmentation methods and achieves performance competitive with state-of-the-art fully supervised models like DeepLab on four X-ray segmentation tasks.
• With additional unlabeled data, CTN reaches the performance level of fully supervised methods without requiring extra annotations.
• After incorporating human-in-the-loop feedback via partial contour corrections, CTN surpasses all fully supervised methods, especially on the HD metric, demonstrating the effectiveness of the feedback mechanism.
• The model maintains high inference speed, with 15.39 FPS, exceeding the fluoroscopic imaging rate (15 FPS), indicating potential for real-time clinical use.
• The contour bending loss effectively regularizes shape evolution, making the model sensitive to acute shape changes and reducing irregular segmentation.
• The contour perceptual loss, computed in ImageNet-pretrained VGG feature space, captures salient structural patterns and is robust to low-level image variations.