[论文解读] Advancing Medical Imaging with Language Models: A Journey from N-grams to ChatGPT

Mingzhe Hu, Shaoyan Pan+2 more|arXiv (Cornell University)|Apr 11, 2023

Artificial Intelligence in Healthcare and Education被引用 20

一句话总结

本文回顾了包括 ChatGPT 在内的语言模型如何被用于提升医疗影像任务，如描述生成、报告生成、要点提取、以及视觉问答，并讨论对临床工作流程的潜在收益。

ABSTRACT

In this paper, we aimed to provide a review and tutorial for researchers in the field of medical imaging using language models to improve their tasks at hand. We began by providing an overview of the history and concepts of language models, with a special focus on large language models. We then reviewed the current literature on how language models are being used to improve medical imaging, emphasizing different applications such as image captioning, report generation, report classification, finding extraction, visual question answering, interpretable diagnosis, and more for various modalities and organs. The ChatGPT was specially highlighted for researchers to explore more potential applications. We covered the potential benefits of accurate and efficient language models for medical imaging analysis, including improving clinical workflow efficiency, reducing diagnostic errors, and assisting healthcare professionals in providing timely and accurate diagnoses. Overall, our goal was to bridge the gap between language models and medical imaging and inspire new ideas and innovations in this exciting area of research. We hope that this review paper will serve as a useful resource for researchers in this field and encourage further exploration of the possibilities of language models in medical imaging.

研究动机与目标

• 提供关于语言模型历史与概念的概述，重点关注大型语言模型。
• 回顾当前文献中应用于医学影像任务（跨模态与器官）的语言模型。
• 将 ChatGPT 作为探索医学影像中新应用的聚焦点。
• 讨论对临床工作流效率、诊断准确性与及时诊断的潜在收益。

提出的方法

• 对语言模型从 N-grams 到大型语言模型的历史发展进行调查。
• 回顾在医学影像中的应用文献，如描述生成、报告生成、报告分类、要点提取、视觉问答与可解释诊断。
• 突出 ChatGPT 并讨论其在医学影像研究中可能的应用途径。
• 综合见解以弥合语言模型与医学影像实践之间的差距。

实验结果

研究问题

• RQ1 语言模型目前如何在医学影像任务（如描述生成、报告、要点提取、VQA、解释）中跨模态与器官应用？
• RQ2 语言模型在临床工作流效率和诊断准确性方面对医学影像有哪些潜在收益与限制？
• RQ3 ChatGPT 为医学影像研究与实践带来哪些特殊角色或应用？
• RQ4 研究人员如何利用语言模型启发医学影像中的新思路与创新？

主要发现

• 语言模型正在用于提升多种医学影像任务，包括描述生成、报告生成、报告分类、要点提取、视觉问答，以及跨各种模态与器官的可解释诊断。
• 将 ChatGPT 突出为研究人员探索医学影像中更多潜在应用的工具。
• 准确高效的语言模型有潜力提升临床工作流程效率、降低诊断错误，并支持医疗专业人员做出及时且准确的诊断。