[Paper Review] Segmenting Potentially Cancerous Areas in Prostate Biopsies using Semi-Automatically Annotated Data

Nikolay Burlutskiy, Nicolas Pinchaud+8 more|arXiv (Cornell University)|Apr 15, 2019

AI in cancer detection14 references4 citations

TL;DR

This paper proposes a deep learning framework for segmenting potentially cancerous areas in prostate biopsies using a novel, objectively defined 'glandular tissue without basal cells' (WOB) class as ground truth. By leveraging semi-automatically generated WOB annotations from immunofluorescence images, the method achieves high accuracy—F1-score of 0.80 and PR AUC of 0.89—on needle biopsies, outperforming pathologists in detecting missed WOB regions and matching human-level performance on H&E-stained slides.

ABSTRACT

Gleason grading specified in ISUP 2014 is the clinical standard in staging prostate cancer and the most important part of the treatment decision. However, the grading is subjective and suffers from high intra and inter-user variability. To improve the consistency and objectivity in the grading, we introduced glandular tissue WithOut Basal cells (WOB) as the ground truth. The presence of basal cells is the most accepted biomarker for benign glandular tissue and the absence of basal cells is a strong indicator of acinar prostatic adenocarcinoma, the most common form of prostate cancer. Glandular tissue can objectively be assessed as WOB or not WOB by using specific immunostaining for glandular tissue (Cytokeratin 8/18) and for basal cells (Cytokeratin 5/6 + p63). Even more, WOB allowed us to develop a semi-automated data generation pipeline to speed up the tremendously time consuming and expensive process of annotating whole slide images by pathologists. We generated 295 prostatectomy images exhaustively annotated with WOB. Then we used our Deep Learning Framework, which achieved the $2^{nd}$ best reported score in Camelyon17 Challenge, to train networks for segmenting WOB in needle biopsies. Evaluation of the model on 63 needle biopsies showed promising results which were improved further by finetuning the model on 118 biopsies annotated with WOB, achieving F1-score of 0.80 and Precision-Recall AUC of 0.89 at the pixel-level. Then we compared the performance of the model against 17 biopsies annotated independently by 3 pathologists using only H\&E staining. The comparison demonstrated that the model performed on a par with the pathologists. Finally, the model detected and accurately outlined existing WOB areas in two biopsies incorrectly annotated as totally WOB-free biopsies by three pathologists and in one biopsy by two pathologists.

Motivation & Objective

• To reduce subjectivity and inter/intra-observer variability in Gleason grading by introducing an objective, biomarker-based ground truth class.
• To develop a scalable, semi-automated pipeline for generating pixel-level annotations of WOB regions in prostate whole slide images.
• To train deep learning models on WOB-annotated data to detect potentially cancerous areas in needle biopsies with high accuracy.
• To evaluate the model’s performance against expert pathologists using standard H&E-stained slides, ensuring clinical relevance.
• To demonstrate that the model can detect WOB regions missed by pathologists, improving diagnostic consistency.

Proposed method

• Introduced the WOB (glandular tissue without basal cells) class as an objective ground truth, defined by co-staining with Cytokeratin 8/18 (glandular tissue) and Cytokeratin 5/6 + p63 (basal cells).
• Developed a semi-automated data generation pipeline to produce H&E images with aligned WOB ground truth masks from scanned immunofluorescence images.
• Trained deep learning models using a framework that achieved 2nd place in the Camelyon17 challenge, fine-tuning on 118 biopsies after pre-training on 295 prostatectomy slides.
• Applied data augmentation and model ensembling to improve robustness and generalization across diverse scanners and clinical labs.
• Used precision-recall AUC and F1-score as primary evaluation metrics on 63 biopsies and compared predictions with independent pathologist annotations on 17 biopsies.
• Conducted ablation studies comparing models trained only on prostatectomies, only on biopsies, or with joint pre-training and fine-tuning to identify optimal training strategy.

Experimental results

Research questions

• RQ1 Can the WOB class serve as a reliable, objective ground truth for training deep learning models in prostate cancer detection?
• RQ2 Does semi-automated annotation generation significantly reduce pathologist workload while maintaining annotation quality for WOB regions?
• RQ3 Can deep learning models trained on WOB-annotated data achieve performance comparable to expert pathologists on H&E-stained needle biopsies?
• RQ4 Can the model detect WOB regions that were missed by multiple expert pathologists in clinically diagnosed WOB-free biopsies?
• RQ5 Does joint pre-training on prostatectomy data followed by fine-tuning on biopsy data improve model generalization and performance?

Key findings

• The model trained on prostatectomy data and fine-tuned on biopsy data (M¹,²ₚᵣ,ᵦᵢ) achieved an F1-score of 0.80 and a precision-recall AUC of 0.89 on 63 needle biopsies.
• The model outperformed models trained only on biopsies or only on prostatectomies, with the best-performing compound model showing the highest robustness across thresholds.
• In a direct comparison with three pathologists, the model matched human-level performance in sensitivity, specificity, and F1-score on 17 H&E-stained biopsies.
• The model successfully detected and accurately outlined WOB regions in two biopsies that were incorrectly labeled as WOB-free by all three pathologists, and in one biopsy missed by two pathologists.
• Semi-automated data generation reduced reliance on manual pathologist annotation while enabling high-quality, exhaustive pixel-level labeling for 295 prostatectomy slides.
• The compound model architecture (pre-training on prostatectomies + fine-tuning on biopsies) significantly improved receptive field and generalization, outperforming alternative training strategies.