[Paper Review] Graph Cross Networks with Vertex Infomax Pooling

Maosen Li, Siheng Chen+2 more|arXiv (Cornell University)|Oct 5, 2020

Advanced Graph Neural Networks54 references4 citations

TL;DR

This paper proposes Graph Cross Networks (GXN), a novel graph neural network architecture that enables comprehensive multiscale feature learning through a trainable vertex infomax pooling (VIPool) mechanism and a cross-shaped feature-crossing layer. VIPool selects informative vertices by maximizing mutual information between vertex and neighborhood features, while the feature-crossing layer enhances information flow across scales, achieving state-of-the-art performance with a 2.12% average accuracy gain on graph classification and 1.15% on vertex classification.

ABSTRACT

We propose a novel graph cross network (GXN) to achieve comprehensive feature learning from multiple scales of a graph. Based on trainable hierarchical representations of a graph, GXN enables the interchange of intermediate features across scales to promote information flow. Two key ingredients of GXN include a novel vertex infomax pooling (VIPool), which creates multiscale graphs in a trainable manner, and a novel feature-crossing layer, enabling feature interchange across scales. The proposed VIPool selects the most informative subset of vertices based on the neural estimation of mutual information between vertex features and neighborhood features. The intuition behind is that a vertex is informative when it can maximally reflect its neighboring information. The proposed feature-crossing layer fuses intermediate features between two scales for mutual enhancement by improving information flow and enriching multiscale features at hidden layers. The cross shape of the feature-crossing layer distinguishes GXN from many other multiscale architectures. Experimental results show that the proposed GXN improves the classification accuracy by 2.12% and 1.15% on average for graph classification and vertex classification, respectively. Based on the same network, the proposed VIPool consistently outperforms other graph-pooling methods.

Motivation & Objective

• To address the limitation of fixed-scale feature learning in existing GNNs, which may overlook local or global graph structures.
• To enable effective information flow across multiple graph scales by designing a trainable, information-theoretic pooling mechanism.
• To develop a novel architecture that fuses intermediate features across scales, rather than only combining them at the final layer.
• To provide direct vertex-to-vertex correspondence across scales, overcoming the limitations of vertex-grouping or global selection methods.
• To achieve superior performance on graph classification and vertex classification tasks through multiscale feature enrichment.

Proposed method

• Proposes Vertex Infomax Pooling (VIPool), a differentiable graph pooling method that selects vertices based on neural estimation of mutual information between vertex features and their neighborhood features.
• Uses a mutual information objective to identify vertices that best represent their local graph structure, ensuring informative and structured coarsening.
• Introduces a cross-shaped feature-crossing layer that enables bidirectional feature exchange between adjacent scales at each network layer.
• Employs a pyramid-structured GXN architecture with multiple parallel scales, where each scale processes graph features using graph convolutional layers.
• Supports both graph pooling and unpooling via VIPool, enabling end-to-end training with direct scale-to-scale vertex correspondence.
• Evaluates three graph structure pooling variants—edge-removal, Kron reduction, and cluster-connection—finding cluster-connection offers the best trade-off between accuracy and efficiency.

Experimental results

Research questions

• RQ1 Can a trainable graph pooling mechanism that maximizes mutual information between vertex and neighborhood features improve multiscale graph representation learning?
• RQ2 Does intermediate feature crossing across multiple scales lead to better performance than late fusion or skip connections in graph neural networks?
• RQ3 How does the proposed GXN architecture compare to existing multiscale GNNs in terms of accuracy and efficiency on graph and vertex classification tasks?
• RQ4 What is the impact of different graph structure pooling operations (e.g., edge-removal, Kron reduction, cluster-connection) on model performance and training time?
• RQ5 To what extent does the selection ratio of vertices in VIPool affect model accuracy and robustness across diverse graph datasets?

Key findings

• GXN achieves an average improvement of 2.12% in graph classification accuracy and 1.15% in vertex classification accuracy over state-of-the-art methods.
• VIPool consistently outperforms other graph pooling methods across multiple datasets, including Cora and IMDB-B, due to its information-theoretic vertex selection.
• The cluster-connection variant of graph structure pooling provides the best balance between accuracy and computational efficiency, outperforming edge-removal and Kron reduction.
• Experiments show that intermediate feature crossing across scales significantly enhances performance, with higher gains when more layers participate in cross-fusion.
• When selecting fewer than 60% of vertices, the greedy vertex selection algorithm in VIPool yields significantly higher mutual information (C value) than the Top-K method, indicating superior selection quality.
• The model maintains strong performance across diverse graph types, with consistent improvements observed on both citation and social network datasets.