[Paper Review] Histographs: Graphs in Histopathology

Shrey Gadiya, Deepak Anand+1 more|arXiv (Cornell University)|Aug 14, 2019

AI in cancer detection7 references4 citations

TL;DR

This paper proposes Histographs—spatial graphs modeling histopathology slides by representing cell nuclei as vertices and inter-nuclear distances as edges—enabling graph convolutional networks (GCNs) to classify cancerous vs. non-cancerous and in situ vs. invasive breast cancer with accuracy competitive to Inception-v3, achieving 93% and 95% respectively on the BACH dataset using fewer parameters.

ABSTRACT

Spatial arrangement of cells of various types, such as tumor infiltrating lymphocytes and the advancing edge of a tumor, are important features for detecting and characterizing cancers. However, convolutional neural networks (CNNs) do not explicitly extract intricate features of the spatial arrangements of the cells from histopathology images. In this work, we propose to classify cancers using graph convolutional networks (GCNs) by modeling a tissue section as a multi-attributed spatial graph of its constituent cells. Cells are detected using their nuclei in H&E stained tissue image, and each cell's appearance is captured as a multi-attributed high-dimensional vertex feature. The spatial relations between neighboring cells are captured as edge features based on their distances in a graph. We demonstrate the utility of this approach by obtaining classification accuracy that is competitive with CNNs, specifically, Inception-v3, on two tasks-cancerous versus non-cancerous and in situ versus invasive-on the BACH breast cancer dataset.

Motivation & Objective

• To address the limitation of CNNs in explicitly capturing spatial arrangements of cells in histopathology images.
• To develop a graph-based representation that integrates micro-level nuclear features and macro-level spatial relationships.
• To demonstrate that GCNs applied to Histographs can achieve competitive classification performance compared to patch-based CNNs.
• To enable interpretable, multi-scale analysis of tissue architecture for improved cancer diagnosis.

Proposed method

• Cells are detected via a fully convolutional network on H&E-stained whole-slide images, with nuclei serving as vertices in a Histograph.
• Vertex features are derived from local image descriptors capturing nuclear morphology and chromatin patterns.
• Edge features are computed based on Euclidean distances between neighboring nuclei, encoding spatial relationships.
• A spatial graph convolutional network (GCN) is trained on the Histograph to classify the entire slide, using robust spatial filtering for heterogeneous graphs.
• The framework processes entire tissue sections as single graphs, avoiding patch aggregation used in conventional CNNs.
• The model is trained end-to-end in a supervised manner on the BACH dataset for two classification tasks.

Experimental results

Research questions

• RQ1 Can a graph-based representation of histopathology slides capture both micro- and macro-structural features relevant to cancer diagnosis?
• RQ2 Does a GCN applied to Histographs achieve competitive classification accuracy compared to state-of-the-art CNNs on whole-slide images?
• RQ3 Can Histographs and GCNs effectively distinguish between in situ and invasive carcinomas, which differ primarily in tissue architecture?
• RQ4 Is the GCN approach more robust or generalizable than hand-crafted graph features for histopathology classification?

Key findings

• The GCN on Histographs achieved 93% classification accuracy on the cancerous vs. non-cancerous task, matching the performance of Inception-v3.
• For the more challenging in situ vs. invasive carcinoma classification, the GCN achieved 95% accuracy, slightly outperforming Inception-v3’s 94%.
• The GCN model used only 18 million trainable parameters, compared to Inception-v3’s 23 million, indicating higher parameter efficiency.
• The GCN directly classifies the entire slide as a single graph, eliminating the need for patch-level aggregation strategies used in CNNs.
• The Histograph framework enables interpretable analysis by potentially highlighting sub-graphs responsible for classification decisions.
• The approach demonstrates the feasibility of using geometric deep learning on histopathology data by modeling tissue architecture as a multi-attributed graph.