[论文解读] Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering

Ye Liu, Shaika Chowdhury+3 more|arXiv (Cornell University)|Aug 6, 2020

Topic Modeling参考文献 48被引用 11

一句话总结

本文提出 MurKe，一种用于 HeadQA 数据集上复杂医疗问答任务的多步推理框架，结合知识抽取技术。通过迭代式检索支持文档、利用检索到的证据重构查询，并应用文本蕴涵模型，MurKe 在强基线模型之上实现了更优性能，展现出在有限训练数据下处理多跳推理时更高的可解释性与鲁棒性。

ABSTRACT

Healthcare question answering assistance aims to provide customer healthcare information, which widely appears in both Web and mobile Internet. The questions usually require the assistance to have proficient healthcare background knowledge as well as the reasoning ability on the knowledge. Recently a challenge involving complex healthcare reasoning, HeadQA dataset, has been proposed, which contains multiple-choice questions authorized for the public healthcare specialization exam. Unlike most other QA tasks that focus on linguistic understanding, HeadQA requires deeper reasoning involving not only knowledge extraction, but also complex reasoning with healthcare knowledge. These questions are the most challenging for current QA systems, and the current performance of the state-of-the-art method is slightly better than a random guess. In order to solve this challenging task, we present a Multi-step reasoning with Knowledge extraction framework (MurKe). The proposed framework first extracts the healthcare knowledge as supporting documents from the large corpus. In order to find the reasoning chain and choose the correct answer, MurKe iterates between selecting the supporting documents, reformulating the query representation using the supporting documents and getting entailment score for each choice using the entailment model. The reformulation module leverages selected documents for missing evidence, which maintains interpretability. Moreover, we are striving to make full use of off-the-shelf pre-trained models. With less trainable weight, the pre-trained model can easily adapt to healthcare tasks with limited training samples. From the experimental results and ablation study, our system is able to outperform several strong baselines on the HeadQA dataset.

研究动机与目标

• 解决复杂、多跳医疗问答任务的挑战，其中答案需要跨多份文档进行推理。
• 提升 HeadQA 数据集上的性能，该数据集因检索线索稀疏且需要深层医学知识而极具挑战性。
• 开发一种可解释的推理框架，通过文档检索与查询重构追踪证据来源。
• 在医疗问答标注数据有限的背景下，有效利用预训练模型并仅进行最小程度的微调。

提出的方法

• 该框架采用两阶段流程：首先，通过维基百科的词元级与语义级检索相结合的方式，检索相关文档。
• 随后，应用迭代式查询重构，利用检索到的文档优化问题表征并识别缺失的证据。
• 通过文本蕴涵模型对每个答案选项与增强后的问题表征进行评分，以选择正确答案。
• 系统采用注意力机制突出文档中的相关证据，确保推理路径的可解释性。
• 将现成的预训练模型（如 BERT）与最小程度的微调相结合，以在有限标注样本下适应医疗问答任务。
• 迭代循环交替进行文档检索、查询重构与蕴涵评分，逐步优化推理过程。

实验结果

研究问题

• RQ1 多步迭代式框架是否能提升在单步检索失效的复杂医疗问答任务上的性能？
• RQ2 结合词元级与语义级检索在大规模语料库中缩小相关文档范围方面效果如何？
• RQ3 利用检索到的证据进行查询重构，在提升答案蕴涵能力与模型可解释性方面能发挥多大作用？
• RQ4 在标注数据有限的情况下，预训练模型能否通过最小微调有效适应医疗问答任务？

主要发现

• MurKe 在 HeadQA 数据集上超越了多个强基线模型，显著优于随机猜测与现有方法。
• 模型在每道问题约 30 份支持文档时达到峰值性能，此后因噪声增加性能略有下降。
• 在 6,765 个问题中仅有 21 个问题的单份文档同时包含问题与答案的关键术语，凸显多步推理的必要性。
• 将图像特征与问题嵌入融合可提升性能，尤其在监督设置下，预训练图像编码器能增强跨模态对齐。
• 通过案例研究验证了系统的可解释性，显示清晰的证据路径，其中文档相关性与推理步骤均可追溯且具有上下文依据。
• 消融实验确认，迭代式检索与重构组件至关重要，移除后性能显著下降。