[Paper Review] Composition-based Multi-Relational Graph Convolutional Networks

Shikhar Vashishth, Soumya Sanyal+2 more|arXiv (Cornell University)|Nov 8, 2019

Advanced Graph Neural NetworksComputer Science39 references136 citations

TL;DR

CompGCN jointly learns node and relation embeddings for multi-relational graphs by composing neighbor nodes with relation embeddings, achieving superior link prediction results and scalable relation handling, with added code release.

ABSTRACT

Graph Convolutional Networks (GCNs) have recently been shown to be quite successful in modeling graph-structured data. However, the primary focus has been on handling simple undirected graphs. Multi-relational graphs are a more general and prevalent form of graphs where each edge has a label and direction associated with it. Most of the existing approaches to handle such graphs suffer from over-parameterization and are restricted to learning representations of nodes only. In this paper, we propose CompGCN, a novel Graph Convolutional framework which jointly embeds both nodes and relations in a relational graph. CompGCN leverages a variety of entity-relation composition operations from Knowledge Graph Embedding techniques and scales with the number of relations. It also generalizes several of the existing multi-relational GCN methods. We evaluate our proposed method on multiple tasks such as node classification, link prediction, and graph classification, and achieve demonstrably superior results. We make the source code of CompGCN available to foster reproducible research.

Motivation & Objective

• Motivate and model multi-relational graphs where edges have labels and directions (e.g., knowledge graphs).
• Develop a GCN framework that jointly embeds nodes and relations to address over-parameterization in relational GCNs.
• Leverage composition operations from knowledge graph embedding to incorporate relation information efficiently.
• Demonstrate effectiveness across link prediction, node classification, and graph classification tasks.

Proposed method

• Represent relations with d-dimensional embeddings and apply relation-aware composition phi(e_s, e_r) to neighbor messages.
• Use direction-specific filters for original, inverse, and self-loop edges via W_lambda(r) (W_O, W_I, W_S).
• Transform relation embeddings with a separate matrix W_rel to align into the node embedding space.
• Incorporate a basis-decomposition strategy for relations to scale with the number of relations (relation basis vectors B).
• Prove that CompGCN generalizes Kipf-GCN, Relational-GCN, Directed-GCN, and Weighted-GCN (Proposition 4.1).
• Provide a stacked k-layer formulation for node and relation embeddings (Eq. 5) and show how to extend to multiple layers.

Experimental results

Research questions

• RQ1 How does CompGCN perform on link prediction compared to existing relational GCNs and KG embeddings?
• RQ2 What is the impact of different graph convolution encoders and composition operators on link prediction performance?
• RQ3 Does CompGCN scale with increasing numbers of relations via basis vectors, while staying parameter-efficient?
• RQ4 How does CompGCN perform on node classification and graph classification tasks relative to strong baselines?

Key findings

• CompGCN outperforms all baselines on 4 of 5 metrics for FB15k-237 and on 3 of 5 metrics for WN18RR in link prediction.
• Using ConvE as the score function with CompGCN (Corr) yields the best overall performance in link prediction settings.
• CompGCN learns both entity and relation embeddings jointly, improving expressiveness over methods that only learn node embeddings.
• A parameter-efficient variant with B relation basis vectors can match full-relations performance, and scales better than Relational-GCN as the number of relations grows.
• Across node and graph classification, CompGCN achieves notable improvements, with 3% average gain on node classification datasets and comparable or improved performance on graph classification.