[Paper Review] Outlining and Filling: Hierarchical Query Graph Generation for Answering Complex Questions over Knowledge Graphs

Yongrui Chen, Huiying Li+3 more|arXiv (Cornell University)|Nov 1, 2021

Topic Modeling4 citations

TL;DR

This paper proposes a hierarchical query graph generation model for complex knowledge graph question answering (KGQA), using a novel graph grammar (AQG) to represent SPARQL queries and enabling end-to-end autoregressive decoding in two stages: outlining the query structure first, then filling in entities and relations. The method significantly reduces search space and avoids local ambiguities, achieving state-of-the-art performance on three complex KGQA benchmarks, especially with pre-trained models.

ABSTRACT

Query graph construction aims to construct the correct executable SPARQL on the KG to answer natural language questions. Although recent methods have achieved good results using neural network-based query graph ranking, they suffer from three new challenges when handling more complex questions: 1) complicated SPARQL syntax, 2) huge search space, and 3) locally ambiguous query graphs. In this paper, we provide a new solution. As a preparation, we extend the query graph by treating each SPARQL clause as a subgraph consisting of vertices and edges and define a unified graph grammar called AQG to describe the structure of query graphs. Based on these concepts, we propose a novel end-to-end model that performs hierarchical autoregressive decoding to generate query graphs. The high-level decoding generates an AQG as a constraint to prune the search space and reduce the locally ambiguous query graph. The bottom-level decoding accomplishes the query graph construction by selecting appropriate instances from the preprepared candidates to fill the slots in the AQG. The experimental results show that our method greatly improves the SOTA performance on complex KGQA benchmarks. Equipped with pre-trained models, the performance of our method is further improved, achieving SOTA for all three datasets used.

Motivation & Objective

• To address the limitations of existing Learn-to-Rank methods in handling complex SPARQL queries with nested subqueries, large search spaces, and local ambiguities.
• To redefine query graph representation to support complex SPARQL syntax, including nested queries and aggregate functions.
• To reduce the search space and avoid local ambiguity in candidate query graph generation through hierarchical, structure-constrained decoding.
• To improve zero-shot and few-shot generalization in KGQA by leveraging structural constraints and pre-trained language models.

Proposed method

• Introduce a new formal grammar called AQG (Answer Query Graph) that treats each SPARQL clause as a subgraph with vertices and edges, enabling representation of complex SPARQL constructs like nested queries and aggregates.
• Propose a two-stage hierarchical autoregressive decoding process: first generating the high-level AQG structure (outlining), then filling slots with specific entities, relations, or values (filling).
• Use a graph grammar-based decoding strategy that allows dynamic insertion of semantic components at any point during generation, avoiding rigid tree-structured constraints.
• Leverage pre-trained language models to enhance the representation of NLQs and improve generalization, especially in low-resource settings.
• Apply structural constraints from the outlined AQG to prune the search space and reduce the likelihood of generating locally correct but globally incorrect query graphs.
• Train an end-to-end model using cross-entropy loss on the hierarchical generation process, with beam search for decoding.

Figure 1 : Examples of the challenges. (a) depicts a complex SPARQL program with a nested query in ComplexWebQuesions [ 12 ] . (b) illustrates a local ambiguity. The green and gold boxes represent the correct and incorrect query graphs, and the dashed boxes represent their structures. For clarity, w

Experimental results

Research questions

• RQ1 Can a hierarchical generation framework with structural constraints reduce the search space and improve accuracy in complex KGQA?
• RQ2 How effective is the proposed AQG grammar in representing complex SPARQL queries, including nested subqueries and aggregate functions?
• RQ3 To what extent does hierarchical decoding mitigate local ambiguity in candidate query graph generation?
• RQ4 Can the model achieve state-of-the-art performance on complex KGQA benchmarks without relying on predefined templates or conditions?

Key findings

• The proposed method achieves state-of-the-art performance on all three major complex KGQA benchmarks: WebQSP, ComplexWebQuestions, and CSQA, especially when enhanced with pre-trained models.
• The hierarchical generation strategy reduces the search space significantly, improving both efficiency and accuracy, particularly on queries with nested subqueries.
• The model outperforms existing Learn-to-Rank baselines by a large margin, especially on queries with complex SPARQL syntax such as those involving FILTER clauses and aggregates.
• The use of AQG grammar enables accurate representation of complex SPARQL constructs that traditional query graphs cannot model, such as nested subqueries.
• The model demonstrates strong generalization ability, with performance gains observed even in few-shot and zero-shot settings when using pre-trained language models.
• The inference time is efficient, with HGNet achieving the best efficiency on both training and inference phases, particularly due to reduced search space in the outlining stage.

Figure 2 : Examples of generalized triples, query graphs and abstract query graphs. Two dashed boxes of the same color indicate that they correspond. In (b) and (c), the number in parentheses after each vertex name indicates its segment number. The red circle, ?x’(0) in (b), is the vertex representi