[论文解读] DDXPlus: A New Dataset For Automatic Medical Diagnosis

Arsène Fansi Tchango, Rishab Goel+3 more|arXiv (Cornell University)|May 18, 2022

Machine Learning in Healthcare被引用 8

一句话总结

DDXPlus 引入了一个大规模的合成数据集，包含130万名患者的结构化病史，涵盖二元、分类和多选症状/前因，以及真实病理结果和鉴别诊断。研究证明，在鉴别诊断上进行自动诊断和症状检测系统训练，可显著提高临床推理效率和模型可解释性，优于仅基于真实病理结果训练的模型。

ABSTRACT

There has been a rapidly growing interest in Automatic Symptom Detection (ASD) and Automatic Diagnosis (AD) systems in the machine learning research literature, aiming to assist doctors in telemedicine services. These systems are designed to interact with patients, collect evidence about their symptoms and relevant antecedents, and possibly make predictions about the underlying diseases. Doctors would review the interactions, including the evidence and the predictions, collect if necessary additional information from patients, before deciding on next steps. Despite recent progress in this area, an important piece of doctors' interactions with patients is missing in the design of these systems, namely the differential diagnosis. Its absence is largely due to the lack of datasets that include such information for models to train on. In this work, we present a large-scale synthetic dataset of roughly 1.3 million patients that includes a differential diagnosis, along with the ground truth pathology, symptoms and antecedents for each patient. Unlike existing datasets which only contain binary symptoms and antecedents, this dataset also contains categorical and multi-choice symptoms and antecedents useful for efficient data collection. Moreover, some symptoms are organized in a hierarchy, making it possible to design systems able to interact with patients in a logical way. As a proof-of-concept, we extend two existing AD and ASD systems to incorporate the differential diagnosis, and provide empirical evidence that using differentials as training signals is essential for the efficiency of such systems or for helping doctors better understand the reasoning of those systems.

研究动机与目标

• 解决医疗人工智能研究中缺乏包含鉴别诊断的大规模数据集的问题。
• 通过将鉴别诊断作为训练信号，提升自动诊断（AD）和自动症状检测（ASD）系统的效率和可解释性。
• 在人工智能驱动的患者互动中，实现更符合临床实际的分层和多模态数据收集。
• 为训练和评估反映真实世界临床推理过程的AD/ASD系统提供基准。
• 通过建模诊断不确定性来支持可信赖、医生可审计的人工智能系统开发。

提出的方法

• 使用专有的医疗知识库和商业自动诊断系统，合成生成数据集，以确保临床合理性。
• 数据集包含49种病理、110种症状和113种前因，症状和前因被分类为二元、分类或多选类型，以实现高效的数据收集。
• 症状按分层结构组织，以支持人工智能代理进行逻辑性、逐步推进的提问。
• 每位患者均生成鉴别诊断，基于症状和前因生成合理的疾病列表，模拟真实临床推理过程。
• 扩展了两种现有的AD/ASD系统（AARLC和BASD），在训练过程中引入鉴别诊断信号。
• 基于证据收集的完整性、鉴别诊断质量以及与临床推理的一致性，对模型性能进行评估。

实验结果

研究问题

• RQ1 在鉴别诊断上训练AD和ASD系统，是否能提升证据收集的质量和完整性？
• RQ2 将鉴别诊断作为训练信号，如何影响模型的可解释性以及与临床工作流程的一致性？
• RQ3 使用分层和多选症状是否能提高人工智能驱动的医疗问诊中症状获取的效率？
• RQ4 与仅基于真实病理结果训练的模型相比，基于鉴别诊断训练的模型在诊断推理和覆盖范围方面表现如何？
• RQ5 配备鉴别诊断的合成数据集能否作为推进临床人工智能系统发展的可行基准？

主要发现

• 在鉴别诊断上训练的模型（如BASD）生成的鉴别诊断比仅基于真实病理结果训练的模型更全面且更具临床相关性。
• 在鉴别诊断信号下训练的AARLC模型收集了更广泛的证据，并探索了更多潜在病理，从而提高了诊断覆盖范围。
• 仅基于真实病理结果训练的AARLC模型生成了不完整的鉴别诊断，并提出了更少的问题，限制了诊断信心。
• 在鉴别诊断上训练的BASD模型收集了足够的证据，但特异性有限，表明需要补充训练信号。
• 联合训练（同时使用鉴别诊断和真实病理结果）取得了最佳效果，实现了证据收集的广度与特异性的平衡。
• 本研究证实，鉴别诊断对于构建高效、可信且与临床一致的医疗诊断人工智能系统至关重要。