[Paper Review] EasierPath: An Open-source Tool for Human-in-the-loop Deep Learning of Renal Pathology

Zheyu Zhu, Yuzhe Lu+4 more|arXiv (Cornell University)|Jul 28, 2020

AI in cancer detection16 references4 citations

TL;DR

EasierPath is an open-source, human-in-the-loop tool that integrates pathologists and deep learning for efficient, large-scale glomerular detection in renal whole slide images (WSI). By leveraging iterative refinement with CircleNet object detection, threshold-based filtering, and seamless annotation compatibility, it reduced annotation effort by 57% and improved average precision from 0.504 to 0.620 after two loops.

ABSTRACT

Considerable morphological phenotyping studies in nephrology have emerged in the past few years, aiming to discover hidden regularities between clinical and imaging phenotypes. Such studies have been largely enabled by deep learning based image analysis to extract sparsely located targeting objects (e.g., glomeruli) on high-resolution whole slide images (WSI). However, such methods need to be trained using labor-intensive high-quality annotations, ideally labeled by pathologists. Inspired by the recent "human-in-the-loop" strategy, we developed EasierPath, an open-source tool to integrate human physicians and deep learning algorithms for efficient large-scale pathological image quantification as a loop. Using EasierPath, physicians are able to (1) optimize the recall and precision of deep learning object detection outcomes adaptively, (2) seamlessly support deep learning outcomes refining using either our EasierPath or prevalent ImageScope software without changing physician's user habit, and (3) manage and phenotype each object with user-defined classes. As a user case of EasierPath, we present the procedure of curating large-scale glomeruli in an efficient human-in-the-loop fashion (with two loops). From the experiments, the EasierPath saved 57 % of the annotation efforts to curate 8,833 glomeruli during the second loop. Meanwhile, the average precision of glomerular detection was leveraged from 0.504 to 0.620. The EasierPath software has been released as open-source to enable the large-scale glomerular prototyping. The code can be found in https://github.com/yuankaihuo/EasierPath

Motivation & Objective

• Address the high labor cost and expertise demands of manual annotation in renal pathology by integrating human and machine intelligence.
• Reduce the time and effort required to curate glomeruli in high-resolution whole slide images (WSI) for large-scale studies.
• Enable pathologists to efficiently refine deep learning detection results using familiar tools like ImageScope while maintaining data integrity.
• Support customizable classification and management of detected glomeruli for downstream phenotyping and model retraining.
• Develop a scalable, open-source pipeline for iterative improvement of glomerular detection through human-in-the-loop feedback loops.

Proposed method

• Employ CircleNet for initial deep learning-based glomerular detection on whole slide images (WSI), outputting bounding circles with confidence scores.
• Apply global thresholding in EasierPath to filter detections by confidence score, balancing recall and precision before manual QA.
• Allow pathologists to perform local quality assurance (QA) and correction using a custom annotation interface or native ImageScope compatibility.
• Extract and manage each detected glomerulus as an individual image with user-defined classification labels.
• Use curated data from each loop to retrain the CircleNet model, forming a closed-loop refinement system.
• Support iterative refinement through two distinct loops: first for biopsy data, second for nephrectomy data, with model retraining after each.

Experimental results

Research questions

• RQ1 Can a human-in-the-loop framework significantly reduce the annotation effort required for large-scale glomerular detection in renal WSI?
• RQ2 To what extent does iterative human feedback improve the performance of deep learning models in glomerular detection?
• RQ3 How does the integration of pathologists with deep learning tools affect the precision and recall of glomerular detection?
• RQ4 Can an open-source tool maintain compatibility with widely used commercial software like ImageScope without disrupting clinical workflows?
• RQ5 What is the impact of curated data from multiple loops on the generalization and accuracy of glomerular detection models?

Key findings

• The EasierPath framework reduced manual annotation effort by 57% in the second loop, saving approximately 2.5 hours of work per human nephrectomy image.
• Average precision (AP) of glomerular detection improved from 0.504 (after first loop) to 0.620 (after second loop), representing a 22.6% relative improvement.
• At an IOU threshold of 0.5, average precision increased by 25.51%, and for small glomeruli, AP improved by 46.28% after the second loop.
• Recall at a precision of 0.8 increased from 0.68 (Loop 1) to 0.90 (Loop 2), a 32.35% improvement, indicating better detection of true glomeruli.
• The precision-recall curves demonstrated consistent gains in both precision and recall after the second loop, confirming enhanced model robustness.
• The closed-loop system enabled effective model retraining using curated data, resulting in measurable performance gains across all AP metrics.