[Paper Review] Recent Advances in the Applications of Convolutional Neural Networks to Medical Image Contour Detection

Zizhao Zhang, Fuyong Xing+3 more|arXiv (Cornell University)|Aug 24, 2017

Medical Image Segmentation Techniques197 references11 citations

TL;DR

This paper reviews recent advances in using convolutional neural networks (CNNs) for medical image contour detection, emphasizing end-to-end deep learning architectures that improve edge and contour accuracy over traditional methods. It identifies key challenges like edge discontinuity, limited medical data, and shape constraints, and proposes solutions through transfer learning, multi-scale feature fusion, and shape priors, significantly enhancing segmentation performance in clinical imaging.

ABSTRACT

The fast growing deep learning technologies have become the main solution of many machine learning problems for medical image analysis. Deep convolution neural networks (CNNs), as one of the most important branch of the deep learning family, have been widely investigated for various computer-aided diagnosis tasks including long-term problems and continuously emerging new problems. Image contour detection is a fundamental but challenging task that has been studied for more than four decades. Recently, we have witnessed the significantly improved performance of contour detection thanks to the development of CNNs. Beyond purusing performance in existing natural image benchmarks, contour detection plays a particularly important role in medical image analysis. Segmenting various objects from radiology images or pathology images requires accurate detection of contours. However, some problems, such as discontinuity and shape constraints, are insufficiently studied in CNNs. It is necessary to clarify the challenges to encourage further exploration. The performance of CNN based contour detection relies on the state-of-the-art CNN architectures. Careful investigation of their design principles and motivations is critical and beneficial to contour detection. In this paper, we first review recent development of medical image contour detection and point out the current confronting challenges and problems. We discuss the development of general CNNs and their applications in image contours (or edges) detection. We compare those methods in detail, clarify their strengthens and weaknesses. Then we review their recent applications in medical image analysis and point out limitations, with the goal to light some potential directions in medical image analysis. We expect the paper to cover comprehensive technical ingredients of advanced CNNs to enrich the study in the medical image domain.

Motivation & Objective

• To analyze the current state of the art in CNN-based contour detection for medical images.
• To identify persistent challenges such as edge discontinuity, lack of labeled data, and shape inconsistency in medical images.
• To evaluate the role of deep learning architectures and training strategies in improving contour detection performance.
• To propose technical directions for integrating shape priors, transfer learning, and multi-scale feature learning in medical contour detection.
• To bridge the gap between general computer vision advances and their application in medical image analysis

Proposed method

• Utilizes end-to-end CNNs for dense pixel-wise prediction of contour maps, replacing patch-based inference with full-image processing.
• Employs transfer learning by fine-tuning pre-trained CNNs on natural image datasets to overcome limited medical training data.
• Introduces multi-scale feature fusion by connecting shallow and deep layers to enhance edge detection and preserve fine details.
• Explores the integration of shape priors through loss functions or structured output prediction to maintain contour continuity.
• Investigates the use of recurrent neural networks (RNNs) and advanced memory networks to model long-range dependencies in fragmented contours.
• Proposes deep supervision mechanisms to improve training stability and feature learning in deeper layers

Experimental results

Research questions

• RQ1 How do end-to-end CNNs improve contour detection performance compared to traditional patch-based methods in medical imaging?
• RQ2 What are the main limitations of standard CNNs in detecting discontinuous or overlapping contours in medical images?
• RQ3 To what extent can transfer learning from natural image datasets improve contour detection in low-data medical imaging scenarios?
• RQ4 How can shape priors be effectively integrated into CNN training to enhance contour continuity and structural plausibility?
• RQ5 What role do multi-scale and skip-connection architectures play in preserving fine-grained edge details in medical image contour detection?

Key findings

• End-to-end CNNs significantly outperform conventional CNNs and even human annotators in contour detection accuracy on medical images.
• Transfer learning from natural image datasets improves performance, especially when medical data is scarce, by leveraging shared low-level features in early layers.
• Multi-scale feature fusion, particularly skip connections between shallow and deep layers, enhances detection of fine edges and improves contour continuity.
• RNNs and memory networks show promise in reconstructing broken or incomplete contours by modeling long-range dependencies in edge sequences.
• Incorporating shape priors into the loss function or structured output layer helps maintain anatomical plausibility and reduces segmentation errors.
• Despite progress, challenges remain in handling overlapping cells, low SNR images, and limited annotation, requiring further research in weakly supervised and few-shot learning