[Paper Review] Rethinking Attention-Based Multiple Instance Learning for Whole-Slide Pathological Image Classification: An Instance Attribute Viewpoint

Linghan Cai, Shenjin Huang+3 more|arXiv (Cornell University)|Mar 30, 2024

AI in cancer detection4 citations

TL;DR

This paper proposes Attribute-Driven MIL (AttriMIL), a novel multiple instance learning framework for whole-slide image (WSI) classification that replaces unreliable attention scores with a learnable attribute scoring mechanism to better quantify instance contributions. By introducing spatial and ranking constraints to model intra- and inter-slide instance correlations, AttriMIL improves tissue discrimination and tumor localization, outperforming SOTA methods on three public benchmarks with enhanced interpretability and OOD detection capability.

ABSTRACT

Multiple instance learning (MIL) is a robust paradigm for whole-slide pathological image (WSI) analysis, processing gigapixel-resolution images with slide-level labels. As pioneering efforts, attention-based MIL (ABMIL) and its variants are increasingly becoming popular due to the characteristics of simultaneously handling clinical diagnosis and tumor localization. However, the attention mechanism exhibits limitations in discriminating between instances, which often misclassifies tissues and potentially impairs MIL performance. This paper proposes an Attribute-Driven MIL (AttriMIL) framework to address these issues. Concretely, we dissect the calculation process of ABMIL and present an attribute scoring mechanism that measures the contribution of each instance to bag prediction effectively, quantifying instance attributes. Based on attribute quantification, we develop a spatial attribute constraint and an attribute ranking constraint to model instance correlations within and across slides, respectively. These constraints encourage the network to capture the spatial correlation and semantic similarity of instances, improving the ability of AttriMIL to distinguish tissue types and identify challenging instances. Additionally, AttriMIL employs a histopathology adaptive backbone that maximizes the pre-trained model's feature extraction capability for collecting pathological features. Extensive experiments on three public benchmarks demonstrate that our AttriMIL outperforms existing state-of-the-art frameworks across multiple evaluation metrics. The implementation code is available at https://github.com/MedCAI/AttriMIL.

Motivation & Objective

• To address the unreliability of attention scores in distinguishing tissue types in attention-based MIL for WSI classification.
• To improve instance discrimination by quantifying the contribution of each patch to bag-level prediction through a novel attribute scoring mechanism.
• To model spatial and semantic correlations among patches using spatial attribute and ranking constraints for better generalization.
• To enable out-of-distribution (OOD) detection by leveraging the interpretability of attribute scores.
• To develop a histopathology-adapted backbone that enhances feature extraction from WSIs.

Proposed method

• Proposes an attribute scoring mechanism that quantifies each instance’s contribution to bag prediction, replacing attention scores as the primary indicator of instance relevance.
• Introduces a spatial attribute constraint to model local spatial correlations within a single WSI by encouraging neighboring patches with similar attributes to have consistent scores.
• Develops an attribute ranking loss to enforce semantic similarity across slides by ranking positive instances higher than negative ones across different WSIs.
• Employs a histopathology-adaptive backbone pre-trained on pathological data to improve feature representation quality for patch-level embeddings.
• Uses a learnable aggregation function that sums attribute scores across instances, with a threshold of 0 to determine bag-level prediction.
• Applies a multi-branch classification head where each branch corresponds to a diagnostic class, and the final prediction is based on whether the aggregated attribute score exceeds 0.

Figure 1: Illustration of ABMIL and our AttriMIL. For features, red cuboids denote positive attributes and blue cuboids are negative; for scores, redder colors represent higher scores and bluer colors indicate lower ones. Notably, for a positive bag, existing methods [ 6 , 7 , 8 , 4 , 5 ] generally

Experimental results

Research questions

• RQ1 Can attribute scores provide a more reliable measure of instance contribution than attention scores in WSI classification?
• RQ2 How can spatial and inter-slide correlations among patches be effectively modeled to improve instance discrimination?
• RQ3 Can the attribute scoring mechanism enable out-of-distribution (OOD) sample detection in pathological image analysis?
• RQ4 Does the integration of a histopathology-adapted backbone improve feature representation and classification performance?
• RQ5 To what extent does AttriMIL improve tumor localization accuracy compared to existing ABMIL-based methods?

Key findings

• AttriMIL outperforms state-of-the-art methods on three public WSI benchmarks, including Camelyon16, PatchCamelyon, and UniToPatho, across multiple evaluation metrics such as AUC and F1-score.
• The attribute scoring mechanism significantly improves positive instance localization, especially for small or subtle tumor regions, as shown in qualitative comparisons.
• AttriMIL effectively distinguishes negative instances embedded in tumor areas—such as pulmonary acini in LUAD and connective tissues in LUSC—demonstrating superior discrimination capability.
• The attribute ranking loss enables the model to learn meaningful instance orderings across slides, enhancing generalization and robustness.
• AttriMIL achieves effective OOD detection by identifying samples with all branch scores below 0, demonstrating its potential for clinical safety in diagnostic systems.
• Ablation studies confirm that both the spatial attribute constraint and attribute ranking loss contribute significantly to performance gains, especially in complex tissue regions.

Figure 2: Overview of AttriMIL framework. For an input WSI, AttriMIL crops it into patches and adopts a histopathology adaptive backbone to extract instance features. Afterward, AttriMIL generates instance attribute scores in each subtype branch (tumor and normal in the tumor detection task) using a