[Paper Review] Modeling Global and Local Node Contexts for Text Generation from Knowledge Graphs

Leonardo F. R. Ribeiro, Yue Zhang+2 more|arXiv (Cornell University)|Jan 29, 2020

Topic Modeling48 references65 citations

TL;DR

This paper proposes unified graph-to-text models that combine global and local node contexts using Graph Attention Networks, achieving state-of-the-art BLEU on AGENDA and WebNLG with CGE-CGE-LW variants.

ABSTRACT

Recent graph-to-text models generate text from graph-based data using either global or local aggregation to learn node representations. Global node encoding allows explicit communication between two distant nodes, thereby neglecting graph topology as all nodes are directly connected. In contrast, local node encoding considers the relations between neighbor nodes capturing the graph structure, but it can fail to capture long-range relations. In this work, we gather both encoding strategies, proposing novel neural models which encode an input graph combining both global and local node contexts, in order to learn better contextualized node embeddings. In our experiments, we demonstrate that our approaches lead to significant improvements on two graph-to-text datasets achieving BLEU scores of 18.01 on AGENDA dataset, and 63.69 on the WebNLG dataset for seen categories, outperforming state-of-the-art models by 3.7 and 3.1 points, respectively.

Motivation & Objective

• Investigate the limitations of global versus local node encodings for graph-to-text generation.
• Propose a unified framework that combines global and local node contexts to learn richer node embeddings.
• Compare cascaded and parallel integration architectures and explore layer-wise integration.
• Evaluate the proposed models on the AGENDA and WebNLG KG-to-text benchmarks.
• Analyze ablations to identify key components driving performance.

Proposed method

• Represent each KG as a multi-relational graph transformed to token-level nodes.
• Implement a global encoder based on Graph Attention Networks (GAT) and a local encoder with relational weights and GRU-based combining.
• Propose four integration architectures: Parallel Graph Encoding (PGE), Cascaded Graph Encoding (CGE), Layer-Wise PGE (PGE-LW), and Layer-Wise CGE (CGE-LW).
• Use a Transformer-like decoder with multi-head attention over encoder outputs and previously generated tokens; apply length penalty for multi-sentence outputs.
• Train with negative log-likelihood, label smoothing, and Byte-Pair Encoding; share vocabulary between nodes and target tokens; tune hyperparameters on development sets.

Experimental results

Research questions

• RQ1 Does combining global and local node contexts improve KG-to-text generation compared to using either alone?
• RQ2 Which integration architecture (PGE, CGE, PGE-LW, CGE-LW) yields the best performance across datasets?
• RQ3 How do graph size and topology affect model performance and the relative benefits of global vs local encodings?
• RQ4 What is the impact of Levi graph transformations on WebNLG results?
• RQ5 How do ablated components (global/local attention, GRU, relation weights) influence performance?

Key findings

• CGE-LW achieves BLEU 18.01 on AGENDA and 63.69 on WebNLG (seen categories), outperforming baselines by 3.7 and 3.1 BLEU points respectively.
• Local encoding plus global encoding improves over baselines, with global context aiding learning across disjoint components in incomplete KGs.
• Layer-wise integration (LW) variants often outperform non-LW configurations; cascaded models generally deliver strong results.
• Levi graph variants improve WebNLG results (63.69 BLEU, 44.47 METEOR, 76.66 CHRF++).
• Ablation shows global attention and FFN contribute significantly to performance; removing local attention or the local encoder markedly reduces BLEU.
• Performance on WebNLG improves with Levi-graph inputs, while AGENDA benefits from cascading global then local processing.