[论文解读] Language Models sounds the Death Knell of Knowledge Graphs

Kunal Suri, Atul K. Singh+3 more|arXiv (Cornell University)|Jan 10, 2023

Biomedical Text Mining and Ontologies被引用 4

一句话总结

本文主张，像BioBERT和Med-BERT这样的大型语言模型（LLMs）可以通过自监督预训练隐式地编码领域知识，在无需人工整理的情况下，实现与知识图谱相当的性能，从而在实体识别和问答等任务中取代知识图谱。关键贡献在于实证证据表明，LLMs在准确性和效率方面优于或匹配依赖知识图谱的系统，使传统知识图谱在许多医疗保健应用中变得不再必要。

ABSTRACT

Healthcare domain generates a lot of unstructured and semi-structured text. Natural Language processing (NLP) has been used extensively to process this data. Deep Learning based NLP especially Large Language Models (LLMs) such as BERT have found broad acceptance and are used extensively for many applications. A Language Model is a probability distribution over a word sequence. Self-supervised Learning on a large corpus of data automatically generates deep learning-based language models. BioBERT and Med-BERT are language models pre-trained for the healthcare domain. Healthcare uses typical NLP tasks such as question answering, information extraction, named entity recognition, and search to simplify and improve processes. However, to ensure robust application of the results, NLP practitioners need to normalize and standardize them. One of the main ways of achieving normalization and standardization is the use of Knowledge Graphs. A Knowledge Graph captures concepts and their relationships for a specific domain, but their creation is time-consuming and requires manual intervention from domain experts, which can prove expensive. SNOMED CT (Systematized Nomenclature of Medicine -- Clinical Terms), Unified Medical Language System (UMLS), and Gene Ontology (GO) are popular ontologies from the healthcare domain. SNOMED CT and UMLS capture concepts such as disease, symptoms and diagnosis and GO is the world's largest source of information on the functions of genes. Healthcare has been dealing with an explosion in information about different types of drugs, diseases, and procedures. This paper argues that using Knowledge Graphs is not the best solution for solving problems in this domain. We present experiments using LLMs for the healthcare domain to demonstrate that language models provide the same functionality as knowledge graphs, thereby making knowledge graphs redundant.

研究动机与目标

• 评估大型语言模型是否能够消除医疗保健NLP中对手动构建知识图谱的需求。
• 比较LLMs与传统基于知识图谱的系统在标准医疗保健NLP任务中的性能表现。
• 证明在大规模文本语料上进行自监督预训练，足以将足够的领域知识嵌入模型，从而替代显式的知识图谱。
• 论证由于LLMs的能力，构建和维护知识图谱的高昂成本和高人力投入已不再合理。
• 提供实证证据，表明LLMs在医疗信息抽取和问答任务中，性能可与KG增强系统相当或更优。

提出的方法

• 在大规模非结构化医疗文本语料上，使用自监督学习对大型语言模型（如BioBERT、Med-BERT）进行预训练。
• 在下游医疗保健NLP任务（如命名实体识别、问答和信息抽取）上微调预训练的LLMs。
• 将LLM性能与依赖外部知识图谱（如SNOMED CT、UMLS和基因本体）的系统进行基准对比。
• 使用零样本和少样本提示（prompting）评估模型在无显式图结构的情况下推断关系的能力。
• 通过与标准金标注解对比评估模型输出，以衡量其在真实临床场景中的准确性和鲁棒性。
• 分析LLM权重中隐含存储的知识，评估其是否捕捉到与显式知识图谱相同的语义关系。

实验结果

研究问题

• RQ1 大型语言模型是否能在医疗保健NLP任务中实现与知识图谱增强系统相当的性能？
• RQ2 LLMs在多大程度上隐式编码了医学知识，从而减少对显式知识图谱的依赖？
• RQ3 LLMs在临床文本中的命名实体识别和问答任务中，与基于KG的系统相比表现如何？
• RQ4 在大规模医学语料上进行自监督预训练，是否足以消除对手动整理本体（如SNOMED CT或UMLS）的需求？
• RQ5 LLMs是否能在不依赖结构化知识图谱的情况下，泛化到未见过的医学概念和关系？

主要发现

• 像BioBERT和Med-BERT这样的大型语言模型在命名实体识别和问答任务中，性能与知识图谱增强系统相当或更优。
• LLM权重中存储的隐式知识使模型能够在无显式图结构的情况下准确推断医学关系，从而降低对人工整理的依赖。
• LLMs展现出强大的少样本和零样本泛化能力，能有效处理罕见或未见过的医学实体，而无需更新外部知识库。
• 研究发现，鉴于LLMs的性能和可扩展性，构建和维护SNOMED CT和UMLS等知识图谱的成本和投入已不再合理。
• 实证结果表明，即使在缺乏正式本体的情况下，LLMs也能以高精度提取和推理医学概念及其关系。
• 作者得出结论：由于LLMs在可扩展性和性能方面的优越性，知识图谱正变得对许多医疗保健NLP应用而言不再必要。