[Paper Review] R-GCN: The R Could Stand for Random

Vic Degraeve, Gilles Vandewiele+2 more|arXiv (Cornell University)|Mar 4, 2022

Advanced Graph Neural Networks4 citations

TL;DR

This paper proposes RR-GCN, a variant of Relational Graph Convolutional Networks (R-GCN) that replaces learned relation-specific transformations with randomly initialized, fixed matrices, demonstrating that message passing via random weights can achieve performance on par with or better than fully trained R-GCNs in node classification and link prediction tasks. The key insight is that the structural inductive bias of KGs and the message passing mechanism itself are more critical than learned parameters.

ABSTRACT

The inception of the Relational Graph Convolutional Network (R-GCN) marked a milestone in the Semantic Web domain as a widely cited method that generalises end-to-end hierarchical representation learning to Knowledge Graphs (KGs). R-GCNs generate representations for nodes of interest by repeatedly aggregating parameterised, relation-specific transformations of their neighbours. However, in this paper, we argue that the the R-GCN's main contribution lies in this "message passing" paradigm, rather than the learned weights. To this end, we introduce the "Random Relational Graph Convolutional Network" (RR-GCN), which leaves all parameters untrained and thus constructs node embeddings by aggregating randomly transformed random representations from neighbours, i.e., with no learned parameters. We empirically show that RR-GCNs can compete with fully trained R-GCNs in both node classification and link prediction settings.

Motivation & Objective

• To investigate whether learned parameters in R-GCNs are essential for effective knowledge graph embedding.
• To evaluate the performance of a fully random message passing mechanism on KGs, independent of training.
• To establish a strong, efficient baseline for future KG embedding methods.
• To explore whether the structural inductive bias of KGs alone suffices for high-quality representations.
• To enable faster experimentation by eliminating backpropagation and weight training.

Proposed method

• RR-GCN replaces all relation-specific transformation matrices in R-GCN with randomly initialized, fixed matrices that are frozen throughout inference.
• Node representations are updated via message passing: each node aggregates messages from neighbors using the random transformation matrices, followed by averaging and ReLU activation.
• The message passing process is applied across multiple layers, with no backpropagation or parameter updates.
• Embeddings are generated in a single forward pass, making RR-GCN significantly faster than end-to-end trained R-GCNs.
• The method uses the same aggregation and update mechanisms as R-GCN but omits all learned weights.
• The approach is evaluated on nine node classification datasets and one link prediction benchmark, comparing against fully trained R-GCNs.

Experimental results

Research questions

• RQ1 Can random, untrained relation-specific transformations in message passing achieve competitive performance in knowledge graph embedding?
• RQ2 Is the performance of R-GCN primarily due to its message passing mechanism rather than its learned parameters?
• RQ3 Does the structural inductive bias of knowledge graphs alone provide sufficient signal for effective node representations?
• RQ4 Can RR-GCN serve as a strong, fast, and lightweight baseline for future KG embedding methods?
• RQ5 In which cases do trained R-GCNs outperform RR-GCNs, and what does this imply about the model's learning behavior?

Key findings

• RR-GCN achieved node classification accuracy of 91.31% ± 0.24% on the AM dataset after removing low-importance relation types, outperforming trained R-GCNs.
• On link prediction, RR-GCN achieved results statistically indistinguishable from those of fully trained R-GCNs across all metrics.
• RR-GCN trained in under two hours, while trained R-GCNs required days to converge, demonstrating a significant speedup.
• The model preserves semantic structure: entities like continents and neighboring countries form coherent clusters in the embedding space.
• In some cases, RR-GCN outperformed trained R-GCNs, suggesting that the latter often learn to suppress or ignore parts of the graph structure.
• The results indicate that the message passing mechanism and graph topology are more important than learned parameters for representation learning in KGs.