[Paper Review] Natural vs Balanced Distribution in Deep Learning on Whole Slide Images for Cancer Detection

Ismat Ara Reshma, Sylvain Cussat‐Blanc+3 more|arXiv (Cornell University)|Dec 21, 2020

AI in cancer detection38 references4 citations

TL;DR

This study compares natural (imbalanced) and artificially balanced class distributions in deep learning for whole slide image (WSI) cancer detection. Using 10-fold cross-validation, it demonstrates that training on the natural distribution—retaining non-region-of-interest (non-ROI) pixels—reduces false positives while maintaining comparable recall, outperforming balanced distributions across all metrics.

ABSTRACT

The class distribution of data is one of the factors that regulates the performance of machine learning models. However, investigations on the impact of different distributions available in the literature are very few, sometimes absent for domain-specific tasks. In this paper, we analyze the impact of natural and balanced distributions of the training set in deep learning (DL) models applied on histological images, also known as whole slide images (WSIs). WSIs are considered as the gold standard for cancer diagnosis. In recent years, researchers have turned their attention to DL models to automate and accelerate the diagnosis process. In the training of such DL models, filtering out the non-regions-of-interest from the WSIs and adopting an artificial distribution (usually, a balanced distribution) is a common trend. In our analysis, we show that keeping the WSIs data in their usual distribution (which we call natural distribution) for DL training produces fewer false positives (FPs) with comparable false negatives (FNs) than the artificially-obtained balanced distribution. We conduct an empirical comparative study with 10 random folds for each distribution, comparing the resulting average performance levels in terms of five different evaluation metrics. Experimental results show the effectiveness of the natural distribution over the balanced one across all the evaluation metrics.

Motivation & Objective

• To investigate the impact of natural versus artificially balanced class distributions on deep learning performance in whole slide image (WSI) cancer detection.
• To evaluate whether retaining non-region-of-interest (non-ROI) pixels—commonly filtered out—improves model generalization and reduces false positives.
• To challenge the common practice of data balancing in WSI-based deep learning by empirically comparing performance under natural and balanced distributions.
• To provide evidence that non-ROI data, often discarded, can improve model robustness by capturing inter-class similarities in histological structures.
• To guide future WSI training protocols by advocating for the use of natural data distributions to reduce pathologist workload through fewer false positives.

Proposed method

• Collected and preprocessed whole slide images (WSIs) from a metastatic lymph node dataset, with ground-truth annotations for cancer (C), non-cancer (¬C), and non-ROI (O) regions.
• Trained two identical deep learning models (E.a and E.b) using the same architecture, differing only in training data distribution: E.a used natural distribution (imbalanced), E.b used artificially balanced distribution.
• Applied standard data augmentation and transfer learning techniques (e.g., ImageNet pretraining) to ensure fair comparison across models.
• Conducted 10-fold cross-validation to ensure robustness and reduce variance in performance evaluation.
• Evaluated models using five metrics: accuracy, F1-score, precision, recall, and AUC, with focus on precision-recall trade-offs.
• Analyzed failure cases via visual inspection of predictions and ground-truth masks to identify causes of false positives, particularly related to histological structure similarity.

Experimental results

Research questions

• RQ1 Does using the natural, imbalanced class distribution in WSI training lead to better performance than artificially balancing the classes in cancer detection?
• RQ2 To what extent do non-region-of-interest (non-ROI) pixels contribute to reducing false positives in deep learning models for WSI analysis?
• RQ3 Can inter-class similarity in histological structures (e.g., blood, fibrosis) between ROI and non-ROI regions be mitigated by including representative non-ROI examples in training?
• RQ4 Is the commonly adopted practice of data balancing in WSI-based deep learning optimal for minimizing false positives while preserving sensitivity?
• RQ5 How does the performance of models trained on natural distributions compare to balanced ones across multiple evaluation metrics in a real-world WSI setting?

Key findings

• Models trained on the natural distribution (E.a) produced significantly fewer false positives than those trained on the balanced distribution (E.b), while maintaining comparable recall levels.
• At 100% recall, the precision of E.a was substantially higher than the random chance baseline, whereas E.b’s precision was only marginally better than random, indicating poor generalization.
• The standard deviation of performance metrics was consistently lower for E.a across folds, indicating more stable and reliable results with the natural distribution.
• Visual analysis revealed that false positives in E.b were primarily caused by inter-class similarity—regions with blood, fibrosis, or lymphocytes in both ROI and non-ROI classes—due to insufficient representation of such non-ROI patterns during training.
• The inclusion of non-ROI pixels in training helped the model learn to distinguish subtle histological structures, reducing misclassification of benign-looking regions as cancerous.
• The study concludes that filtering out non-ROI regions during data preparation may be counterproductive, as these examples improve model robustness and reduce false positives in clinical deployment.