[Paper Review] Exploiting Edge Features in Graph Neural Networks

Liyu Gong, Qiang Cheng|arXiv (Cornell University)|Sep 7, 2018

Machine Learning in Materials Science10 citations

TL;DR

This paper proposes Edge-Enhanced Graph Neural Networks (EGNN) that improve graph learning by leveraging multi-dimensional edge features through doubly stochastic normalization, adaptive edge feature updates across layers, and generalized attention mechanisms. The method outperforms GCN and GAT on node classification and molecular property prediction, with gains of over 0.2 AUC on Tox21 and 0.3 RMSE improvement on Freesolv.

ABSTRACT

Edge features contain important information about graphs. However, current state-of-the-art neural network models designed for graph learning, e.g. graph convolutional networks (GCN) and graph attention networks (GAT), adequately utilize edge features, especially multi-dimensional edge features. In this paper, we build a new framework for a family of new graph neural network models that can more sufficiently exploit edge features, including those of undirected or multi-dimensional edges. The proposed framework can consolidate current graph neural network models; e.g. graph convolutional networks (GCN) and graph attention networks (GAT). The proposed framework and new models have the following novelties: First, we propose to use doubly stochastic normalization of graph edge features instead of the commonly used row or symmetric normalization approches used in current graph neural networks. Second, we construct new formulas for the operations in each individual layer so that they can handle multi-dimensional edge features. Third, for the proposed new framework, edge features are adaptive across network layers. As a result, our proposed new framework and new models can exploit a rich source of graph information. We apply our new models to graph node classification on several citation networks, whole graph classification, and regression on several molecular datasets. Compared with the current state-of-the-art methods, i.e. GCNs and GAT, our models obtain better performance, which testify to the importance of exploiting edge features in graph neural networks.

Motivation & Objective

• To address the underutilization of multi-dimensional edge features in state-of-the-art GNNs like GCN and GAT.
• To improve graph representation learning by adapting edge features across network layers instead of using a fixed initial adjacency matrix.
• To develop a unified framework that extends GCN and GAT to handle rich, multi-dimensional edge features.
• To enable effective modeling of directed graphs through a novel edge feature design for directed edges.
• To empirically validate the superiority of the proposed framework on node classification and graph-level regression/classification tasks.

Proposed method

• Introduces doubly stochastic normalization of edge features, replacing row or symmetric normalization used in GCN and GAT.
• Proposes a generalized attention mechanism that incorporates multi-dimensional edge features into the attention weight computation.
• Designs a learnable edge feature update mechanism that adapts edge features at each layer, enabling dynamic refinement of edge representations.
• Develops a new message-passing formula that supports multi-dimensional edge features in graph convolution operations.
• Applies exponential linear units (ELU) as activation functions and uses Adam optimizer with L2 regularization for training.
• Constructs edge features for directed edges using a symmetric transformation to preserve directional information in the model.

Experimental results

Research questions

• RQ1 Can multi-dimensional edge features significantly improve performance in graph neural networks compared to binary or one-dimensional edge features?
• RQ2 Does adaptive edge feature learning across layers lead to better representation learning than fixed edge features?
• RQ3 How does doubly stochastic normalization compare to row or symmetric normalization in terms of model stability and performance?
• RQ4 Can the proposed framework effectively handle both undirected and directed graphs with rich edge attributes?
• RQ5 To what extent does incorporating edge features improve performance on node-level and graph-level prediction tasks?

Key findings

• On the Tox21 dataset, EGNN(C) and EGNN(A) achieved an average AUC improvement of over 0.2 compared to the Weave baseline in multi-label classification.
• On the Lipophilicity dataset, EGNN models reduced RMSE by approximately 0.1 compared to the Weave model in regression tasks.
• On the Freesolv dataset, EGNN models achieved an RMSE improvement of about 0.3 over the Weave baseline in regression.
• The performance of EGNN(C) and EGNN(A) was nearly identical across all three molecular datasets, indicating robustness to architectural variations.
• Ablation studies confirmed that each component—doubly stochastic normalization, adaptive edge features, and multi-dimensional attention—contributed significantly to the overall performance gain.
• The proposed framework consistently outperformed both GCN and GAT across all benchmark datasets, demonstrating its generalization and effectiveness.