[论文解读] Sequential Query Encoding For Complex Query Answering on Knowledge Graphs

Jiaxin Bai, Tianshi Zheng+1 more|arXiv (Cornell University)|Feb 25, 2023

Advanced Graph Neural Networks被引用 5

一句话总结

本文提出顺序查询编码（SQE），一种用于知识图谱上复杂查询问答的简化神经方法。与将查询解析为计算图并递归执行参数化操作符的方法不同，SQE将查询线性化为标记序列，并使用类似LSTM或Transformer的序列编码器对其进行编码。SQE在分布内查询上实现了最先进性能，在分布外查询上也表现出相当的泛化能力，表明直接序列编码可优于复杂的操作符执行流水线。

ABSTRACT

Complex Query Answering (CQA) is an important and fundamental task for knowledge graph (KG) reasoning. Query encoding (QE) is proposed as a fast and robust solution to CQA. In the encoding process, most existing QE methods first parse the logical query into an executable computational direct-acyclic graph (DAG), then use neural networks to parameterize the operators, and finally, recursively execute these neuralized operators. However, the parameterization-and-execution paradigm may be potentially over-complicated, as it can be structurally simplified by a single neural network encoder. Meanwhile, sequence encoders, like LSTM and Transformer, proved to be effective for encoding semantic graphs in related tasks. Motivated by this, we propose sequential query encoding (SQE) as an alternative to encode queries for CQA. Instead of parameterizing and executing the computational graph, SQE first uses a search-based algorithm to linearize the computational graph to a sequence of tokens and then uses a sequence encoder to compute its vector representation. Then this vector representation is used as a query embedding to retrieve answers from the embedding space according to similarity scores. Despite its simplicity, SQE demonstrates state-of-the-art neural query encoding performance on FB15k, FB15k-237, and NELL on an extended benchmark including twenty-nine types of in-distribution queries. Further experiment shows that SQE also demonstrates comparable knowledge inference capability on out-of-distribution queries, whose query types are not observed during the training process.

研究动机与目标

• 解决现有查询编码（QE）方法中依赖神经化操作符递归执行所导致的复杂性与潜在过参数化问题。
• 探索是否可使用更简单、统一的序列编码器替代查询编码中的多步操作符参数化与执行流水线。
• 评估查询编码模型在分布内与分布外查询类型上的忠实性与组合泛化能力。
• 研究位置编码设计对Transformer在知识图谱推理中组合泛化的影响。

提出的方法

• SQE使用基于搜索的算法将逻辑查询的计算图线性化为标记序列。
• 对标记序列应用序列编码器（如LSTM或Transformer）以生成密集向量表示（查询嵌入）。
• 利用查询嵌入通过嵌入空间中的相似性评分检索答案实体，避免递归操作符执行。
• 在Transformer中采用相对位置编码（RPE）以提升在分布外查询上的性能。
• 通过端到端联合优化实体嵌入与神经编码器参数进行模型训练。
• 该方法避免显式操作符参数化与执行，以单一统一的编码步骤取而代之。

Figure 1: Three complex query examples and corresponding interpretations expressed in natural language.

实验结果

研究问题

• RQ1 直接序列编码方法是否能在知识图谱查询编码中超越标准的操作符参数化与执行流水线？
• RQ2 SQE在分布内查询上的表现相较于现有最先进查询编码方法如何？
• RQ3 SQE在训练中未见过的分布外查询上的泛化能力有多强？
• RQ4 Transformer中位置编码的设计是否会影响复杂查询问答中的组合泛化？

主要发现

• SQE在FB15k、FB15k-237和NELL上的分布内查询上均实现了最先进性能，优于现有神经查询编码方法。
• SQE在分布外查询上的知识推理能力与先前方法相当，表明其具备强大的组合泛化能力。
• 在Transformer中使用相对位置编码（RPE）显著提升了分布外查询的性能，表明标准位置编码可能抑制泛化能力。
• 结果表明，现有QE方法中递归操作符执行的复杂性可能并不必要，单一序列编码器即可实现更优或相当的结果。
• SQE的简洁性——依赖线性化与序列编码——在无需显式建模集合运算或投影函数的情况下，已证明有效。

Figure 2: (A) The example complex query with the meaning of Finding the substances that interact with the proteins associated with Alzheimer’s or Mad cow disease. (B) The computational graph of the complex query; (C) The linearized computational graph with proper indentations as a sequence of tokens