[Paper Review] ProtoMIL: Multiple Instance Learning with Prototypical Parts for Fine-Grained Interpretability

Dawid Rymarczyk, Aneta Kaczyńska+3 more|arXiv (Cornell University)|Aug 24, 2021

Explainable Artificial Intelligence (XAI)46 references4 citations

TL;DR

ProtoMIL introduces a self-explainable Multiple Instance Learning framework that leverages visual prototypes to enhance both model accuracy and fine-grained interpretability. By modeling prototypical parts through a case-based reasoning process, it enables detailed instance-level explanations while achieving state-of-the-art performance on five benchmark MIL datasets.

ABSTRACT

Multiple Instance Learning (MIL) gains popularity in many real-life machine learning applications due to its weakly supervised nature. However, the corresponding effort on explaining MIL lags behind, and it is usually limited to presenting instances of a bag that are crucial for a particular prediction. In this paper, we fill this gap by introducing ProtoMIL, a novel self-explainable MIL method inspired by the case-based reasoning process that operates on visual prototypes. Thanks to incorporating prototypical features into objects description, ProtoMIL unprecedentedly joins the model accuracy and fine-grained interpretability, which we present with the experiments on five recognized MIL datasets.

Motivation & Objective

• To address the lack of fine-grained interpretability in Multiple Instance Learning (MIL) models.
• To develop a self-explainable MIL framework that provides meaningful, instance-level explanations.
• To integrate visual prototypes into MIL for improved model interpretability without sacrificing performance.
• To bridge the gap between model accuracy and interpretability in weakly supervised learning.

Proposed method

• ProtoMIL employs a prototype-based attention mechanism that learns visual prototypes representing key parts of instances within bags.
• It uses a case-based reasoning approach to compare input instances to learned prototypes for prediction and explanation.
• The model jointly optimizes for classification accuracy and prototype alignment through a contrastive learning objective.
• Prototypes are updated during training to represent discriminative, semantically meaningful parts of the data.
• Attention weights are computed based on similarity between instances and prototypes, enabling instance-level interpretability.
• The framework is trained end-to-end using weak supervision, with bag-level labels only.

Experimental results

Research questions

• RQ1 Can a MIL model achieve both high accuracy and fine-grained interpretability through prototype-based reasoning?
• RQ2 How do visual prototypes improve the model's ability to explain predictions at the instance level?
• RQ3 To what extent does incorporating prototypes enhance generalization and robustness in weakly supervised learning?
• RQ4 How does the case-based reasoning mechanism compare to standard attention in MIL models?

Key findings

• ProtoMIL achieves state-of-the-art performance on five recognized MIL benchmark datasets.
• The model provides fine-grained interpretability by highlighting specific parts of instances that are most similar to learned prototypes.
• The integration of prototypes improves both model accuracy and explanation quality compared to baseline MIL methods.
• Explanations generated by ProtoMIL are more semantically meaningful and localized than those from standard attention mechanisms.