[Paper Review] Relational Graph Attention Networks

Dan Busbridge, Dane Sherburn+2 more|arXiv (Cornell University)|Apr 11, 2019

Advanced Graph Neural NetworksComputer Science38 references127 citations

TL;DR

The paper studies Relational Graph Attention Networks (RGATs), extending graph attention to relational graphs, and compares WIRGAT and ARGAT variants against RGCN and other baselines across transductive and inductive tasks. RGATs show mixed results with marginal gains in some inductive graph classification, but often underperform compared to spectral RGCN depending on task and setup.

ABSTRACT

We investigate Relational Graph Attention Networks, a class of models that extends non-relational graph attention mechanisms to incorporate relational information, opening up these methods to a wider variety of problems. A thorough evaluation of these models is performed, and comparisons are made against established benchmarks. To provide a meaningful comparison, we retrain Relational Graph Convolutional Networks, the spectral counterpart of Relational Graph Attention Networks, and evaluate them under the same conditions. We find that Relational Graph Attention Networks perform worse than anticipated, although some configurations are marginally beneficial for modelling molecular properties. We provide insights as to why this may be, and suggest both modifications to evaluation strategies, as well as directions to investigate for future work.

Motivation & Objective

• Explore whether relational graph attention can improve performance on node classification and graph classification tasks with relational data.
• Assess two RGAT variants (Within-Relation Graph Attention and Across-Relation Graph Attention) under additive and multiplicative logits.
• Compare RGATs to Relational Graph Convolutional Networks (RGCN) under the same conditions and baselines.
• Provide implementations and guidance for fair evaluation and future work in relational graph models.

Proposed method

• Define RGAT layer with intermediate relation-specific representations g_i^(r) = H W^(r).
• Compute logits E_{i,j}^{(r)} using additive or multiplicative attention based on g_i^(r) and g_j^(r).
• Support two softmax schemes: Within-Relation (per relation) and Across-Relation (across relations).
• Aggregate attention-weighted neighbor representations to update node features; support multi-head attention.
• Incorporate basis decomposition to reduce parameter count across relations and heads.
• Evaluate on transductive node classification (AIFB, MUTAG) and inductive graph classification (Tox21) with baselines including FEAT, WL, RDF2Vec, RGCN.

Experimental results

Research questions

• RQ1 Do RGATs outperform or compete with RGCN on transductive node classification tasks?
• RQ2 Do RGAT variants (WIRGAT vs ARGAT) and logit types (additive vs multiplicative) yield gains on transductive tasks?
• RQ3 Are RGATs advantageous for inductive graph classification tasks such as molecular property prediction (Tox21)?
• RQ4 How do different softmax schemes (Within-Relation vs Across-Relation) affect relational attention in practice?
• RQ5 What baselines reveal the strengths and limitations of RGATs across relational datasets?

Key findings

• RGATs show competitive or inferior performance compared to RGCN depending on task and configuration.
• On AIFB (transductive), additive WIRGAT performs best among RGAT variants, with some gains over constant attention; on MUTAG, RGCN often outperforms RGAT.
• On Tox21 (inductive), multiplicative ARGAT and multiplicative WIRGAT offer modest gains over RGCN, with some gains being significant for certain comparisons.
• Generally, WIRGAT with additive logits and ARGAT with multiplicative logits are recommended for different task types, but no RGAT variant universally outperforms RGCN.
• Transductive tasks may favor spectral methods like RGCN or traditional graph kernels (WL), while certain inductive tasks can benefit from RGAT variants, particularly multiplicative ARGAT.
• The study provides a rigorous hyperparameter exploration, CDFs, and statistical tests to support conclusions.