[Paper Review] Explainable Artificial Intelligence for Medical Applications: A Review

Qiyang Sun, Alican Akman+1 more|arXiv (Cornell University)|Nov 15, 2024

Machine Learning in Healthcare4 citations

TL;DR

This review synthesizes explainable AI (XAI) techniques for medical applications, categorizing 19 methods across visual, audio, and multimodal domains to enhance transparency in AI-driven diagnostics. It advocates for user-centered, multimodal, and personalized XAI frameworks to improve clinical trust, fairness, and real-world adoption in healthcare.

ABSTRACT

The continuous development of artificial intelligence (AI) theory has propelled this field to unprecedented heights, owing to the relentless efforts of scholars and researchers. In the medical realm, AI takes a pivotal role, leveraging robust machine learning (ML) algorithms. AI technology in medical imaging aids physicians in X-ray, computed tomography (CT) scans, and magnetic resonance imaging (MRI) diagnoses, conducts pattern recognition and disease prediction based on acoustic data, delivers prognoses on disease types and developmental trends for patients, and employs intelligent health management wearable devices with human-computer interaction technology to name but a few. While these well-established applications have significantly assisted in medical field diagnoses, clinical decision-making, and management, collaboration between the medical and AI sectors faces an urgent challenge: How to substantiate the reliability of decision-making? The underlying issue stems from the conflict between the demand for accountability and result transparency in medical scenarios and the black-box model traits of AI. This article reviews recent research grounded in explainable artificial intelligence (XAI), with an emphasis on medical practices within the visual, audio, and multimodal perspectives. We endeavour to categorise and synthesise these practices, aiming to provide support and guidance for future researchers and healthcare professionals.

Motivation & Objective

• Address the critical need for transparency and accountability in AI-driven medical decision-making due to the black-box nature of complex models.
• Systematically categorize and evaluate 19 XAI techniques across visual, audio, and multimodal medical applications.
• Identify gaps in current XAI research, particularly in audio and multimodal medicine, and propose pathways for improvement.
• Promote the integration of patient and clinician feedback in XAI development to enhance clinical relevance and user trust.
• Advance ethical AI in healthcare by emphasizing fairness, bias mitigation, and diverse, representative datasets.

Proposed method

• Propose a taxonomy of 19 XAI techniques based on four criteria: model type, explanation type, modality, and implementation approach.
• Classify XAI methods using implementation-based categorization, evaluating each against interpretability, fidelity, and clinical usability.
• Review and synthesize recent XAI applications in medical imaging (e.g., CNNs for tumor detection), audio analysis (e.g., acoustic pathology), and multimodal systems (e.g., Transformers integrating images and physiological data).
• Integrate emerging techniques such as sonification—converting data into non-verbal sound forms—to enhance multimodal data explanation.
• Advocate for the use of fairness metrics (e.g., disparate impact, equal opportunity) during model training and interpretation to detect and reduce bias.
• Emphasize the need for user-informed development, including clinician and patient involvement in data collection and explanation design to ensure practicality and acceptance.

Experimental results

Research questions

• RQ1 How can XAI techniques be systematically categorized and evaluated for medical applications across visual, audio, and multimodal domains?
• RQ2 What are the current limitations of XAI in healthcare, particularly in audio and multimodal medicine, and how can they be addressed?
• RQ3 How can XAI models be made more personalized, interactive, and adaptable to individual patient profiles in clinical decision-making?
• RQ4 What role do data diversity and fairness metrics play in reducing bias and improving ethical standards in medical XAI systems?
• RQ5 How can novel explanation methods like sonification enhance the interpretability of complex medical data in multimodal settings?

Key findings

• XAI techniques such as saliency maps, LIME, and attention visualization have shown strong potential in interpreting deep learning models for medical imaging, improving clinician trust and diagnostic accuracy.
• Audio-based medical AI applications remain underexplored in XAI, with limited use of techniques tailored to the temporal and spectral characteristics of acoustic data.
• Multimodal XAI, particularly using Transformers to integrate imaging, audio, and physiological data, enables more comprehensive and context-aware clinical insights.
• Sonification offers a novel, non-visual method for explaining complex medical data, enhancing accessibility and multimodal understanding.
• Current XAI methods often lack rigorous validation in real clinical settings and require deeper integration with clinicians and patients to ensure usability and relevance.
• Bias in medical AI is exacerbated by unrepresentative datasets; fairness assessments and data augmentation techniques are essential to improve equity and reduce disparities.