[论文解读] Towards Sparse Hierarchical Graph Classifiers

Cătălina Cangea, Petar Veličković+3 more|arXiv (Cornell University)|Nov 3, 2018

Advanced Graph Neural Networks参考文献 19被引用 198

一句话总结

论文提出一个可扩展、保留稀疏性的分层图分类器，使用可学习的池化机制在不产生平方记忆的情况下对图进行下采样，在标准基准数据集上与 DiffPool 结果相当。

ABSTRACT

Recent advances in representation learning on graphs, mainly leveraging graph convolutional networks, have brought a substantial improvement on many graph-based benchmark tasks. While novel approaches to learning node embeddings are highly suitable for node classification and link prediction, their application to graph classification (predicting a single label for the entire graph) remains mostly rudimentary, typically using a single global pooling step to aggregate node features or a hand-designed, fixed heuristic for hierarchical coarsening of the graph structure. An important step towards ameliorating this is differentiable graph coarsening---the ability to reduce the size of the graph in an adaptive, data-dependent manner within a graph neural network pipeline, analogous to image downsampling within CNNs. However, the previous prominent approach to pooling has quadratic memory requirements during training and is therefore not scalable to large graphs. Here we combine several recent advances in graph neural network design to demonstrate that competitive hierarchical graph classification results are possible without sacrificing sparsity. Our results are verified on several established graph classification benchmarks, and highlight an important direction for future research in graph-based neural networks.

研究动机与目标

• 通过固定池化或全局池化来激发图分类及其局限性。
• 开发一种可微分、稀疏的池化层，在不产生平方记忆的情况下对图进行下采样。
• 结合卷积、池化和读出，构建端到端的图分类模型。
• 在标准图分类基准数据集上展示可扩展性和有竞争力的性能。

提出的方法

• 使用带自环的感知均值池化传播规则进行图卷积：MP(X,A)=sigma(D^(-1) Â X Θ + X Θ' ).
• 池化层将节点以固定比率 k 下采样，使用投影向量 p 和 top-k 选择形成更稀疏的图：y=Xp/||p||, i=top-k(y,k), X'=X⊙tanh(y) 在下标 i, A'=A_i,i。
• 通过每层的汇总 s^(l) = (1/N^(l)) sum_i x_i^(l) || max_i x_i^(l)，最终图表示为 s = sum_l s^(l)，并使用 MLP 进行预测。
• 训练使用带数据集特定学习率的 Adam，并通过三个卷积-池化块在池化后保持 80% 的节点。
• 与 DiffPool 相比，该方法保持 O(V+E) 的存储而非平方级别，提升可扩展性。

实验结果

研究问题

• RQ1 是否可以在图卷积网络中实现可微分池化而不产生二次记忆成本？
• RQ2 稀疏感知的分层池化在标准图分类基准上是否能保持有竞争力的性能？
• RQ3 所提出的方法在数据集上与 DiffPool 和稀疏聚合基线相比有何差异？
• RQ4 通过 Jumping Knowledge 风格的多尺度读出对分类性能是否有帮助？

主要发现

数据集	模型	Enzymes	D&D	Collab	Proteins
Enzymes	Graphlet	41.03	–	–	–
D&D	Shortest-path	74.85	78.86	76.43	76.43
Collab	1-WL	64.66	59.10	78.61	73.76
Proteins	WL-QA	72.91	–	–	75.26
Enzymes	PatchySAN	–	–	–	–
D&D	GraphSAGE	54.25	75.42	68.25	70.48
Collab	ECC	53.50	74.10	67.79	72.65
Proteins	Set2Set	60.15	78.12	71.75	74.29
Enzymes	SortPool	57.12	79.37	73.76	75.54
D&D	DiffPool-Det	58.33	75.47	82.13	75.62
Collab	DiffPool-NoLP	62.67	79.98	75.63	77.42
Proteins	DiffPool	64.23	81.15	75.50	78.10
Enzymes	Ours	64.17	78.59	74.54	75.46

• 所提出的稀疏池化方法在准确性上具有竞争力，紧贴 DiffPool 的变体，同时避免二次内存使用。
• 在 Enzymes、D&D、Collab 和 Proteins 上，模型优于稀疏 GraphSAGE 基线，在大多数数据集上约落后 DiffPool 变体1个百分点左右。
• 该方法显示出有利的 GPU 内存行为，能够扩展到 DiffPool 需要更高内存的大图。
• 在实验中，Ours 模型在测试方法中通常取得最佳或接近最佳的分数，验证了稀疏友好型分层池化的有效性。