[Paper Review] Hypergraph Convolution and Hypergraph Attention

Song Bai, Feihu Zhang+1 more|arXiv (Cornell University)|Jan 23, 2019

Advanced Graph Neural NetworksComputer Science73 references39 citations

TL;DR

The paper introduces hypergraph convolution and hypergraph attention to extend graph neural networks to high-order relationships, with demonstrated gains on semi-supervised node classification compared to pairwise GNNs.

ABSTRACT

Recently, graph neural networks have attracted great attention and achieved prominent performance in various research fields. Most of those algorithms have assumed pairwise relationships of objects of interest. However, in many real applications, the relationships between objects are in higher-order, beyond a pairwise formulation. To efficiently learn deep embeddings on the high-order graph-structured data, we introduce two end-to-end trainable operators to the family of graph neural networks, i.e., hypergraph convolution and hypergraph attention. Whilst hypergraph convolution defines the basic formulation of performing convolution on a hypergraph, hypergraph attention further enhances the capacity of representation learning by leveraging an attention module. With the two operators, a graph neural network is readily extended to a more flexible model and applied to diverse applications where non-pairwise relationships are observed. Extensive experimental results with semi-supervised node classification demonstrate the effectiveness of hypergraph convolution and hypergraph attention.

Motivation & Objective

• Motivate learning on high-order (non-pairwise) relationships beyond simple graphs.
• Introduce end-to-end trainable hypergraph convolution and hypergraph attention operators.
• Demonstrate improved semi-supervised node classification performance on benchmark datasets.

Proposed method

• Define hypergraph with incidence matrix H, vertex degrees D and hyperedge degrees B.
• Propose hypergraph convolution: X^{(l+1)} = σ(D^{-1/2} H W B^{-1} H^T D^{-1/2} X^{(l)} P) (symmetric normalization) or X^{(l+1)} = σ(D^{-1} H W B^{-1} H^T X^{(l)} P) (asymmetric normalization).
• Prove that standard graph convolution is a special case of hypergraph convolution when hyperedges are pairwise.
• Optionally enable hypergraph attention by learning a dynamic incidence matrix H via attention scores (Eq. 7–8).
• Discuss practical aspects: complexity, sparsity, skip connections, and multi-head extensions (Eq. 13).
• Show how to integrate with existing GNNs and leverage non-pairwise relationships.

Experimental results

Research questions

• RQ1 Can hypergraph convolution generalize graph convolution to higher-order relationships?
• RQ2 Does hypergraph attention provide further performance gains by learning dynamic connectivity?
• RQ3 How do hypergraph operators perform on standard semi-supervised node classification benchmarks compared to GCN/GAT?
• RQ4 What are practical considerations (normalization, sparsity, training stability) for implementing hypergraph operators?

Key findings

• Hypergraph convolution reduces to graph convolution when hyperedges are pairwise and normalized appropriately.
• Hypergraph attention further improves results by learning dynamic connectivity over hyperedges.
• On Cora and Citeseer, hypergraph-based methods outperform their pairwise counterparts (e.g., Hyper-Conv. vs GCN*, Hyper-Atten. vs GAT*).
• On Pubmed, hypergraph attention achieves competitive accuracy with state-of-the-art methods.
• Table-based comparisons show Ours achieves 82.7% (Cora), 71.2% (Citeseer), and 78.4% (Pubmed) in Table 5, closely matching or exceeding several baselines.