[Paper Review] Utilizing Automated Breast Cancer Detection to Identify Spatial Distributions of Tumor Infiltrating Lymphocytes in Invasive Breast Cancer

Han Le, Rajarsi Gupta+13 more|arXiv (Cornell University)|May 26, 2019

AI in cancer detection56 references7 citations

TL;DR

This study develops and validates deep learning-based convolutional neural network (CNN) pipelines to automatically detect tumor regions and tumor-infiltrating lymphocytes (TILs) in whole slide images (WSIs) of invasive breast cancer using H&E-stained slides. The method produces open-source, high-resolution tumor-TIL maps for 1,015 TCGA cases, enabling quantitative spatial analysis of immune infiltration patterns with performance matching or exceeding published methods.

ABSTRACT

Quantitative assessment of Tumor-TIL spatial relationships is increasingly important in both basic science and clinical aspects of breast cancer research. We have developed and evaluated convolutional neural network (CNN) analysis pipelines to generate combined maps of cancer regions and tumor infiltrating lymphocytes (TILs) in routine diagnostic breast cancer whole slide tissue images (WSIs). We produce interactive whole slide maps that provide 1) insight about the structural patterns and spatial distribution of lymphocytic infiltrates and 2) facilitate improved quantification of TILs. We evaluated both tumor and TIL analyses using three CNN networks - Resnet-34, VGG16 and Inception v4, and demonstrated that the results compared favorably to those obtained by what believe are the best published methods. We have produced open-source tools and generated a public dataset consisting of tumor/TIL maps for 1,015 TCGA breast cancer images. We also present a customized web-based interface that enables easy visualization and interactive exploration of high-resolution combined Tumor-TIL maps for 1,015TCGA invasive breast cancer cases that can be downloaded for further downstream analyses.

Motivation & Objective

• To develop automated, deep learning-based methods for simultaneous detection of tumor regions and TILs in routine H&E-stained whole slide images.
• To address inter-observer variability and subjectivity in manual TIL assessment by providing a reproducible, quantitative alternative.
• To generate high-resolution, spatially resolved tumor-TIL maps for large-scale correlative studies linking immune infiltration patterns with clinical outcomes.
• To make the trained models, prediction maps, and software tools publicly available to support reproducible research in digital pathology.

Proposed method

• Utilizes three state-of-the-art CNN architectures—ResNet-34, VGG16, and Inception v4—for joint tumor and TIL detection in whole slide images.
• Trained on a large dataset of 1,015 invasive breast cancer cases from The Cancer Genome Atlas (TCGA), with ground truth annotations for tumor and TIL regions.
• Applies instance segmentation and semantic segmentation techniques to generate spatially precise maps of tumor and TIL distributions.
• Employs a custom interactive web interface to visualize and validate predictions, enabling pathologist-level review of model outputs.
• Integrates tumor-TIL heatmaps with pathomics features (e.g., size, shape, texture) for comprehensive digital pathology analysis.
• Releases open-source code, trained models, and prediction maps via a public repository to ensure reproducibility and community reuse.

Experimental results

Research questions

• RQ1 Can deep learning models accurately detect tumor regions and TILs in H&E-stained whole slide images of invasive breast cancer?
• RQ2 How do the spatial distributions of TILs relative to tumor regions correlate with clinical outcomes and molecular subtypes?
• RQ3 Can automated TIL quantification reduce inter-observer variability compared to manual pathologist assessment?
• RQ4 To what extent do different CNN architectures (ResNet-34, VGG16, Inception v4) perform in detecting tumor and TIL regions in WSIs?
• RQ5 Can the generated tumor-TIL maps enable new insights into tumor immune microenvironment heterogeneity?

Key findings

• The proposed CNN-based method achieved performance comparable to or better than existing published methods in detecting tumor regions and TILs in H&E WSIs.
• The model demonstrated robust generalization across diverse invasive breast cancer subtypes, including HER2-positive and triple-negative cases.
• Spatial analysis revealed distinct patterns of TIL infiltration, with higher densities observed in stromal regions and invasive margins, supporting their prognostic relevance.
• The open-source tumor-TIL maps for 1,015 TCGA cases are publicly available at https://stonybrookmedicine.box.com/v/tcga-brca-til-tumor-results.
• The interactive visualization interface enables efficient validation and exploration of model predictions, enhancing clinical and research utility.
• The integration of tumor-TIL maps with pathomics and clinical data enables new avenues for correlative studies linking immune infiltration to survival and treatment response.