[Paper Review] Large AI Models in Health Informatics: Applications, Challenges, and the Future

Jianing Qiu, Lin Li+12 more|arXiv (Cornell University)|Mar 21, 2023

Artificial Intelligence in Healthcare and Education8 citations

TL;DR

The paper provides a comprehensive review of large AI models (LAMs) and their applications across health informatics, outlining sectors, benefits, challenges, and future directions.

ABSTRACT

Large AI models, or foundation models, are models recently emerging with massive scales both parameter-wise and data-wise, the magnitudes of which can reach beyond billions. Once pretrained, large AI models demonstrate impressive performance in various downstream tasks. A prime example is ChatGPT, whose capability has compelled people's imagination about the far-reaching influence that large AI models can have and their potential to transform different domains of our lives. In health informatics, the advent of large AI models has brought new paradigms for the design of methodologies. The scale of multi-modal data in the biomedical and health domain has been ever-expanding especially since the community embraced the era of deep learning, which provides the ground to develop, validate, and advance large AI models for breakthroughs in health-related areas. This article presents a comprehensive review of large AI models, from background to their applications. We identify seven key sectors in which large AI models are applicable and might have substantial influence, including 1) bioinformatics; 2) medical diagnosis; 3) medical imaging; 4) medical informatics; 5) medical education; 6) public health; and 7) medical robotics. We examine their challenges, followed by a critical discussion about potential future directions and pitfalls of large AI models in transforming the field of health informatics.

Motivation & Objective

• Identify seven key health informatics sectors where large AI models can influence practice and research.
• Summarize current progress and benchmarks of LAMs in health-related tasks across multiple modalities.
• Discuss challenges, risks, and limitations of deploying LAMs in biomedical settings.
• Outline promising future directions and potential pitfalls to guide researchers and practitioners.

Proposed method

• Synthesize recent literature on LAMs including LLMs, LVMs, and LMMs and categorize them by training data modalities and architectures.
• Describe characteristic features of LAMs: large scale, large-scale pre-training, and cross-domain generalization.
• Compare LAMs to prior SOTA methods in biomedical tasks through qualitative discussion and cited examples.
• Summarize domain-specific applications and adaptation strategies like RLHF and prompting techniques in health care contexts.
• Highlight practical considerations such as data availability, annotation burden, and modality integration.

Experimental results

Research questions

• RQ1 What are the seven health informatics sectors where LAMs have the potential to impact practice and research?
• RQ2 How do LAMs perform and adapt across bioscience, diagnostic imaging, informatics, education, public health, and robotics tasks compared to previous methods?
• RQ3 What are the principal challenges, limitations, and risks associated with deploying LAMs in health informatics?
• RQ4 What future directions, methodologies, and safeguards are likely to shape the next decade of LAM development in health care?

Key findings

• LAMs enable cross-modal learning and zero-/few-shot capabilities that can transform diagnosis, imaging, and informatics tasks.
• In bioinformatics, large protein/RNA language models and structure/prediction methods show strong performance and speed advantages, but still rely on data quality and benchmarks for evaluation.
• In medical imaging, LVMs and LVLMs demonstrate zero-shot segmentation and classification capabilities, with adaptation needed for modality-specific data (e.g., MRI, OCT).
• Biomedical LLMs (e.g., Med-PaLM, Med-PaLM 2, GatorTron) show improved medical QA, inference, and discharge summary generation, indicating potential for clinical support and documentation.
• LMMs and multi-modal alignment strategies enhance image-text understanding and enable retrieval and generation tasks in medical contexts.
• Challenges include data availability, annotation costs, calibration, factual grounding, safety, and the risk of over-reliance on automated outputs.