[Paper Review] Responsible AI in Healthcare

Federico Cabitza, Davide Ciucci+2 more|arXiv (Cornell University)|Feb 19, 2022

Misinformation and Its Impacts4 citations

TL;DR

This paper proposes a framework for responsible AI in healthcare by addressing uncertainty in medical data and online health information disorder. It introduces uncertainty-aware machine learning models and explainable, multi-criteria decision systems to improve transparency, fairness, and trustworthiness in clinical decision support.

ABSTRACT

This article discusses open problems, implemented solutions, and future research in the area of responsible AI in healthcare. In particular, we illustrate two main research themes related to the work of two laboratories within the Department of Informatics, Systems, and Communication at the University of Milano-Bicocca. The problems addressed concern, in particular, {uncertainty in medical data and machine advice}, and the problem of online health information disorder.

Motivation & Objective

• Address the challenge of uncertainty in medical data—biological, analytical, and pre/post-analytical variability—commonly overlooked in standard ML pipelines.
• Develop transparent, explainable AI systems that support human decision-making in healthcare, ensuring accountability and reducing bias.
• Combat health information disorder online by creating gradual, non-binary assessments of information genuineness rather than rigid filtering.
• Improve the reliability of health information retrieval through model-driven, explainable evaluation frameworks based on Multi-Criteria Decision Making (MCDM).
• Integrate domain knowledge from medical experts into hybrid, partly model-driven, partly data-driven systems to ensure clinical relevance and ethical robustness.

Proposed method

• Design and implement novel machine learning algorithms that explicitly model and mitigate uncertainty from biological, analytical, and pre/post-analytical sources in clinical data.
• Utilize explainable AI techniques to ensure traceability and interpretability of model outputs, supporting human oversight and decision autonomy.
• Develop a multi-criteria decision-making (MCDM) framework to rank health information by a continuous measure of genuineness, replacing binary classification of 'true/false' or 'trusted/disinformation'.
• Create labeled datasets and evaluation strategies for health information retrieval, validated through participation in CLEF eHealth and other benchmarking initiatives.
• Build open-source tools for uncertainty management (e.g., scikit-weak, uncertainpy) and misinformation detection (e.g., health-misinformation GitHub repo) to support reproducibility and community use.
• Integrate medical domain knowledge into AI systems through collaboration with clinicians, ensuring that model outputs reflect clinical reality and ethical standards.

Experimental results

Research questions

• RQ1 How can machine learning systems be made robust to uncertainty arising from biological, analytical, and pre/post-analytical variability in clinical data?
• RQ2 What role can explainable, multi-criteria decision models play in assessing the genuineness of online health information without resorting to binary filtering?
• RQ3 How can AI systems be designed to preserve user autonomy and avoid over-censorship of legitimate health content in the face of misinformation?
• RQ4 In what ways can hybrid, model-driven and data-driven approaches improve the transparency and clinical relevance of AI in healthcare decision support?
• RQ5 How can evaluation frameworks be standardized to ensure fairness, reproducibility, and auditability of AI systems in health information retrieval?

Key findings

• The proposed uncertainty-aware ML models effectively reduce the impact of data variability, improving reliability in clinical predictions.
• A gradual, non-binary assessment of information genuineness outperforms binary classification in preserving access to legitimate health content while reducing misinformation risks.
• The integration of MCDM techniques into health information evaluation enables explainable, traceable, and auditable decision-making processes.
• Open-source tools such as uncertainpy and scikit-weak provide practical implementations of uncertainty modeling for clinical data, supporting reproducibility.
• Participation in CLEF eHealth and related workshops demonstrates the feasibility and effectiveness of standardized evaluation protocols for consumer health search and misinformation detection.
• Hybrid, knowledge-informed AI models show greater transparency and alignment with clinical reasoning, enhancing trust and accountability in decision support systems.