[Paper Review] A tool for user friendly, cloud based, whole slide image segmentation

Brendon Lutnick, David Manthey+1 more|arXiv (Cornell University)|Jan 18, 2021

AI in cancer detection4 references4 citations

TL;DR

This paper presents a cloud-based, user-friendly tool for whole slide image (WSI) segmentation using a state-of-the-art convolutional neural network, integrated via a graphical interface built on HistomicsTK. It enables non-expert users to perform remote, human-in-the-loop segmentation of WSIs with high accuracy, achieving an F-score > 0.97 in glomeruli segmentation on renal tissue slides.

ABSTRACT

Convolutional neural networks, the state of the art for image segmentation, have been successfully applied to histology images by many computational researchers. However, the translatability of this technology to clinicians and biological researchers is limited due to the complex and undeveloped user interface of the code, as well as the extensive computer setup required. As an extension of our previous work (arXiv:1812.07509), we have developed a tool for segmentation of whole slide images (WSIs) with an easy to use graphical user interface. Our tool runs a state-of-the-art convolutional neural network for segmentation of WSIs in the cloud. Our plugin is built on the open source tool HistomicsTK by Kitware Inc. (Clifton Park, NY), which provides remote data management and viewing abilities for WSI datasets. The ability to access this tool over the internet will facilitate widespread use by computational non-experts. Users can easily upload slides to a server where our plugin is installed and perform human in the loop segmentation analysis remotely. This tool is open source, and has the ability to be adapted to segment of any pathological structure. For a proof of concept, we have trained it to segment glomeruli from renal tissue images, achieving an F-score > 0.97 on holdout tissue slides.

Motivation & Objective

• To address the limited translatability of deep learning-based WSI segmentation tools to clinicians and biologists due to complex interfaces and high computational demands.
• To develop a cloud-hosted, interactive segmentation tool with a simple graphical user interface for non-expert users.
• To enable remote, human-in-the-loop segmentation of whole slide images via a web-based platform.
• To demonstrate the tool’s adaptability to various pathological structures through a proof-of-concept on glomeruli in renal tissue.
• To improve accessibility and usability of state-of-the-art WSI segmentation for biomedical research.

Proposed method

• The tool is built as a plugin for HistomicsTK, an open-source platform for WSI management and visualization.
• It leverages a pre-trained convolutional neural network for semantic segmentation of histology images.
• All processing occurs in the cloud, eliminating local computational requirements for users.
• Users upload WSIs through a web interface, and the system performs inference using the deep learning model.
• The interface supports interactive correction and refinement of segmentation results in real time.
• The system is designed to be extensible, allowing retraining for segmentation of any pathological structure.

Experimental results

Research questions

• RQ1 Can a cloud-based, user-friendly interface enable non-expert biologists and clinicians to perform accurate WSI segmentation?
• RQ2 How does the performance of the proposed tool compare to existing deep learning methods in terms of segmentation accuracy?
• RQ3 To what extent can the tool be adapted to segment different pathological structures beyond glomeruli?
• RQ4 What is the impact of remote, human-in-the-loop interaction on segmentation quality and usability?
• RQ5 Can a web-based platform eliminate the need for local GPU infrastructure in WSI analysis?

Key findings

• The tool achieved an F-score > 0.97 in segmenting glomeruli from renal tissue slides on holdout data, demonstrating high segmentation accuracy.
• The cloud-based architecture allows users to perform segmentation without local GPU or complex software setup.
• The integration with HistomicsTK enables secure remote data management and visualization of WSI datasets.
• The graphical user interface supports interactive segmentation, allowing users to correct and refine results in real time.
• The tool is open source and extensible, enabling adaptation for segmentation of diverse pathological structures.
• The system enables widespread use by computational non-experts through simplified access and intuitive interaction.