[Paper Review] Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology

Andreas Holzinger, Bernd Malle+5 more|arXiv (Cornell University)|Dec 18, 2017

AI in cancer detectionComputer Science32 references69 citations

TL;DR

The paper outlines a research agenda for integrating AI/ML with human pathologists in digital pathology, emphasizing explainability, data integration, and educational as well as clinical workflows to create an “augmented pathologist.”

ABSTRACT

Digital pathology is not only one of the most promising fields of diagnostic medicine, but at the same time a hot topic for fundamental research. Digital pathology is not just the transfer of histopathological slides into digital representations. The combination of different data sources (images, patient records, and *omics data) together with current advances in artificial intelligence/machine learning enable to make novel information accessible and quantifiable to a human expert, which is not yet available and not exploited in current medical settings. The grand goal is to reach a level of usable intelligence to understand the data in the context of an application task, thereby making machine decisions transparent, interpretable and explainable. The foundation of such an "augmented pathologist" needs an integrated approach: While machine learning algorithms require many thousands of training examples, a human expert is often confronted with only a few data points. Interestingly, humans can learn from such few examples and are able to instantly interpret complex patterns. Consequently, the grand goal is to combine the possibilities of artificial intelligence with human intelligence and to find a well-suited balance between them to enable what neither of them could do on their own. This can raise the quality of education, diagnosis, prognosis and prediction of cancer and other diseases. In this paper we describe some (incomplete) research issues which we believe should be addressed in an integrated and concerted effort for paving the way towards the augmented pathologist.

Motivation & Objective

• Motivate the need for AI/ML in digital pathology to augment human expertise rather than replace clinicians.
• Propose an integrated, human-centered approach combining HCI and knowledge discovery/data mining (KDD) to enable explainable insights.
• Identify data-centric prerequisites such as standardization of whole slide imaging (WSI) formats and annotation/metadata.
• Discuss data integration across images, EPRs, and *omics data to unlock new biomarkers and diagnostic insights.
• Highlight methodological directions including interpretable deep learning, graph-based methods, and topological data mining to support explainability.

Proposed method

• Describe a workflow for machine-aided pathology, including hypothesis formulation, feature detection/classification, and risk prediction.
• Discuss multi-resolution analysis and the use of image pyramids to manage terabytes of slide data.
• Review approaches for interpretable deep learning, including visualization techniques like deconvolutional networks to relate features back to input space.
• Suggest graph-theoretic and probabilistic approaches to link heterogeneous data (images, EHRs, *omics) for explainable reasoning.
• Propose topological data mining concepts for handling manifold structures and proximity-based explanations in medical images.

Experimental results

Research questions

• RQ1 What are the key challenges and research issues in making AI-based pathology explainable and usable in clinical practice?
• RQ2 How can integration of imaging, clinical records, and *omics data support an augmented pathologist?
• RQ3 What AI/ML and visualization strategies can make deep learning models in digital pathology more transparent and trustworthy?
• RQ4 How can graph-based and topological data mining approaches enhance cross-modal reasoning in pathology?

Key findings

• AI/ML can augment diagnostic workflows across hypothesis formulation, feature detection, and prognosis, potentially improving quality and education.
• WSI data are extremely large (e.g., 16 Gigapixels per image) and require multi-resolution analysis and data management strategies.
• Interpretable DL approaches (e.g., deconvnet visualizations) can provide insight into learned features across layers, aiding explainability.
• Graph-based representations offer a path to linking image regions with heterogeneous data sources for integrated analysis.
• Data standardization and cross-vendor visualization paradigms are essential for interoperability and education.