[Paper Review] Enhancing AMR-to-Text Generation with Dual Graph Representations

Leonardo F. R. Ribeiro, Claire Gardent+1 more|TUbilio (Technical University of Darmstadt)|Sep 1, 2019

Topic Modeling33 references4 citations

TL;DR

This paper proposes a dual graph representation model for AMR-to-text generation that separately encodes top-down and bottom-up structural perspectives of the input AMR graph using distinct graph encoders. By leveraging complementary node representations and state-of-the-art graph neural networks, the model achieves state-of-the-art performance on two AMR benchmarks, outperforming prior methods including those using syntactic information.

ABSTRACT

Generating text from graph-based data, such as Abstract Meaning Representation (AMR), is a challenging task due to the inherent difficulty in how to properly encode the structure of a graph with labeled edges. To address this difficulty, we propose a novel graph-to-sequence model that encodes different but complementary perspectives of the structural information contained in the AMR graph. The model learns parallel top-down and bottom-up representations of nodes capturing contrasting views of the graph. We also investigate the use of different node message passing strategies, employing different state-of-the-art graph encoders to compute node representations based on incoming and outgoing perspectives. In our experiments, we demonstrate that the dual graph representation leads to improvements in AMR-to-text generation, achieving state-of-the-art results on two AMR datasets.

Motivation & Objective

• To address the challenge of encoding complex, non-Euclidean graph structures in AMR-to-text generation.
• To improve text generation quality by capturing complementary structural perspectives of the AMR graph.
• To eliminate reliance on positional encoding and anonymization by using parallel graph representations.
• To evaluate the effectiveness of different graph encoders in the context of dual representation learning.
• To achieve state-of-the-art results on AMR-to-text generation without requiring additional syntactic information.

Proposed method

• The model constructs two separate graph structures: one for top-down (TD) and one for bottom-up (BU) traversal of the AMR graph.
• Each node in both graphs is encoded using a distinct graph neural network (GNN) encoder, learning representations from incoming (BU) and outgoing (TD) edges.
• The method employs message passing strategies to aggregate neighborhood information from both perspectives, enabling richer node representations.
• The dual representations are combined in a sequence-to-sequence decoder to generate fluent, semantically accurate text.
• The model integrates a copy mechanism to preserve rare entities without requiring delexicalization or anonymization.
• Three GNN variants—GIN, GAT, and GGNN—are evaluated as encoders to compare performance across different message-passing mechanisms.

Experimental results

Research questions

• RQ1 Can dual, parallel encoding of top-down and bottom-up graph structures improve AMR-to-text generation performance?
• RQ2 How do different graph neural network architectures (e.g., GIN, GAT, GGNN) perform when used as encoders in the dual representation framework?
• RQ3 Does the dual representation approach reduce the need for positional encoding or external syntactic features?
• RQ4 To what extent does the model preserve input concepts and avoid generating out-of-vocabulary or hallucinated tokens?
• RQ5 How does the model compare to state-of-the-art baselines, including those using syntactic information?

Key findings

• The proposed dual graph representation model achieves state-of-the-art results on two AMR benchmarks, outperforming previous models including those using syntactic information.
• The G2S-GGNN variant achieved the best performance with a BLEU score of 48.66 and a METEOR score of 50.77 on the LDC2017T10 test set.
• The model reduced the fraction of added tokens (48.66%) and missing tokens (34.06%) compared to the S2S baseline, indicating better semantic adequacy.
• Human evaluation showed that the G2S-GGNN model scored higher in both meaning similarity (mean 78.4) and readability (mean 76.2) than the S2S baseline.
• The dual representation approach successfully captured complex dependencies, such as subject-verb relations in embedded clauses, where previous models failed due to n-gram frequency bias.
• The GIN-based encoder showed the closest performance to the reference (GOLD) in terms of added and missing tokens, suggesting superior coverage of input concepts.