[Paper Review] PanNuke Dataset Extension, Insights and Baselines

Jevgenij Gamper, Navid Alemi Koohbanani+7 more|arXiv (Cornell University)|Mar 24, 2020

AI in cancer detection45 references156 citations

TL;DR

This paper extends PanNuke to ~190k nuclei across 19 tissues, benchmarks segmentation/classification models, analyzes real-world generalization, and provides a semi-automatic, QC-ed ground-truth pipeline with NuClick.

ABSTRACT

The emerging area of computational pathology (CPath) is ripe ground for the application of deep learning (DL) methods to healthcare due to the sheer volume of raw pixel data in whole-slide images (WSIs) of cancerous tissue slides. However, it is imperative for the DL algorithms relying on nuclei-level details to be able to cope with data from `the clinical wild', which tends to be quite challenging. We study, and extend recently released PanNuke dataset consisting of ~200,000 nuclei categorized into 5 clinically important classes for the challenging tasks of segmenting and classifying nuclei in WSIs. Previous pan-cancer datasets consisted of only up to 9 different tissues and up to 21,000 unlabeled nuclei and just over 24,000 labeled nuclei with segmentation masks. PanNuke consists of 19 different tissue types that have been semi-automatically annotated and quality controlled by clinical pathologists, leading to a dataset with statistics similar to the clinical wild and with minimal selection bias. We study the performance of segmentation and classification models when applied to the proposed dataset and demonstrate the application of models trained on PanNuke to whole-slide images. We provide comprehensive statistics about the dataset and outline recommendations and research directions to address the limitations of existing DL tools when applied to real-world CPath applications.

Motivation & Objective

• Provide a larger, clinically representative nuclei dataset with pixel-level boundaries and multi-class labels.
• Describe a semi-automatic ground-truth generation and quality control pipeline leveraging NuClick.
• Benchmark multiple state-of-the-art nuclei segmentation models on PanNuke and establish baselines.
• Analyze generalization of trained models to unseen tissues (e.g., brain) and discuss real-world applicability.
• Offer guidelines and research directions to address limitations of DL tools in real-world computational pathology.

Proposed method

• Semi-automatic ground-truth generation by aggregating multiple public datasets to train an FCNN detector for nuclei, followed by iterative pathology verification.
• Nuclei segmentation masks generated with NuClick conditioned on verified nucleus dots to produce accurate pixel-wise boundaries.
• Patch extraction and three-fold randomized dataset splits to enable fair cross-model comparison across tissues.
• Evaluation using multi-class Panoptic Quality (mPQ) and binary PQ (bPQ) metrics, with IoU threshold 0.5 for true positives.
• Benchmark of four models (DIST, Mask-RCNN, Micro-Net, HoVer-Net) adapted for simultaneous nuclei segmentation and classification; also include a detection-based U-Net variant.
• Generalization test by applying PanNuke-trained models to unseen brain tissue.

Experimental results

Research questions

• RQ1 Can PanNuke provide robust, nuclei-level segmentation and classification across 19 diverse tissues with clinically relevant artifacts?
• RQ2 How do state-of-the-art segmentation models perform on PanNuke using standardized, multi-class PQ metrics?
• RQ3 Does training on PanNuke generalize to unseen tissues, and what are the limitations in terms of nucleus category prediction?
• RQ4 What are the practical implications and limitations of current DL tools when deployed on real-world computational pathology data?

Key findings

• PanNuke contains 189,744 exhaustively annotated nuclei from 20,000+ WSIs across 19 tissues, with 5 nucleus classes.
• HoVer-Net achieves the best average performance (mPQ and bPQ) across tissues for nucleus segmentation and classification.
• Dead nuclei are challenging (small size yields low PQ), and class imbalance affects performance in some categories.
• Segmentation models generally outperform a detection-based approach when evaluated on nucleus centroids for detection metrics.
• Models trained on PanNuke show promising segmentation on an unseen brain tissue, indicating generalization for segmentation but challenging category prediction in a new tissue.
• Providing an evaluation framework (mPQ/bPQ) and releasing evaluation code promotes reproducibility and further benchmarking.