[论文解读] InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization

Fan-Yun Sun, Jordan Hoffmann+2 more|arXiv (Cornell University)|Jul 31, 2019

Advanced Graph Neural Networks参考文献 72被引用 147

一句话总结

InfoGraph 通过最大化图级表示与多尺度子结构之间的互信息来学习固定长度的图嵌入；InfoGraph* 将其扩展为半监督学习，以利用未标注数据。

ABSTRACT

This paper studies learning the representations of whole graphs in both unsupervised and semi-supervised scenarios. Graph-level representations are critical in a variety of real-world applications such as predicting the properties of molecules and community analysis in social networks. Traditional graph kernel based methods are simple, yet effective for obtaining fixed-length representations for graphs but they suffer from poor generalization due to hand-crafted designs. There are also some recent methods based on language models (e.g. graph2vec) but they tend to only consider certain substructures (e.g. subtrees) as graph representatives. Inspired by recent progress of unsupervised representation learning, in this paper we proposed a novel method called InfoGraph for learning graph-level representations. We maximize the mutual information between the graph-level representation and the representations of substructures of different scales (e.g., nodes, edges, triangles). By doing so, the graph-level representations encode aspects of the data that are shared across different scales of substructures. Furthermore, we further propose InfoGraph*, an extension of InfoGraph for semi-supervised scenarios. InfoGraph* maximizes the mutual information between unsupervised graph representations learned by InfoGraph and the representations learned by existing supervised methods. As a result, the supervised encoder learns from unlabeled data while preserving the latent semantic space favored by the current supervised task. Experimental results on the tasks of graph classification and molecular property prediction show that InfoGraph is superior to state-of-the-art baselines and InfoGraph* can achieve performance competitive with state-of-the-art semi-supervised models.

研究动机与目标

• 激发在无监督和半监督设置下表现良好的图级表示的学习。
• 克服手工设计的图核与子树聚焦嵌入的局限性。
• 使表示适用于图分类和分子性质预测。
• 利用多尺度补丁信息捕捉跨图共享的结构。

提出的方法

• 在图的全局表示与其多尺度补丁表示之间最大化互信息。
• 使用 Graph Isomorphism Network (GIN) 编码器获取节点/补丁嵌入，并使用 READOUT 形成全局图表示。
• 跨层将补丁表示连接形成多尺度特征，并使用 Jensen-Shannon MI 估计器进行优化。
• 采用按批次的负采样生成大量正/负样本对，以实现有效对比学习。
• InfoGraph* 引入两个编码器（有监督和无监督），并最大化它们中间表示之间的互信息以进行知识转移。
• 判别器 Tψ 对（全局、补丁）对进行评分以估计表示之间的互信息（Eq. 4）；说明涉及按批次采样和 MI 估计（Eq. 5）。

实验结果

研究问题

• RQ1 在图级表示与子结构级表示之间最大化互信息是否能够产生有效的无监督图嵌入？
• RQ2 引入半监督的学生-教师风格框架（InfoGraph*）是否提升分子性质预测的性能？
• RQ3 在标准基准测试上，InfoGraph 相较于传统图核和先前的无监督图表示方法表现如何？
• RQ4 多尺度补丁表示对图级嵌入质量有何影响？

主要发现

方法	MUTAG	PTC-MR	REDDIT-B	REDDIT-M5K	IMDB-B	IMDB-M
RW	83.72±1.50	57.85±1.30	OMR	OMR	50.68±0.26	34.65±0.19
SP	85.22±2.43	58.24±2.44	64.11±0.14	39.55±0.22	55.60±0.22	37.99±0.30
GK	81.66±2.11	57.26±1.41	77.34±0.18	41.01±0.17	65.87±0.98	43.89±0.38
WL	80.72±3.00	57.97±0.49	68.82±0.41	46.06±0.21	72.30±3.44	46.95±0.46
DGK	87.44±2.72	60.08±2.55	78.04±0.39	41.27±0.18	66.96±0.56	44.55±0.52
MLG	87.94±1.61	63.26±1.48	>1 Day	>1 Day	66.55±0.25	41.17±0.03
node2vec	72.63±10.20	58.58±8.00	-	-	-	-
sub2vec	61.05±15.80	59.99±6.38	71.48±0.41	36.68±0.42	55.26±1.54	36.67±0.83
graph2vec	83.15±9.25	60.17±6.86	75.78±1.03	47.86±0.26	71.1±0.54	50.44±0.87
InfoGraph	89.01±1.13	61.65±1.43	82.50±1.42	53.46±1.03	73.03±0.87	49.69±0.53

• InfoGraph 在 6 个基准数据集中的 4 个上超越了若干最先进的图核，在其他数据集上也保持高度竞争力。
• 在半监督分子性质预测中，InfoGraph* 在 12 个目标中有 11 个获得最佳结果，在大多数情况下超过 Mean Teacher。
• InfoGraph 将无监督的基于互信息的学习与按批次的负采样以及 GIN 编码器结合起来，产生强大的图级表示。
• InfoGraph 在无监督和半监督学习范式下，对图分类和分子性质预测表现具备竞争力。
• 本文讨论了简单联合目标下的负迁移现象，由 InfoGraph* 通过两个编码器和基于 MI 的对齐（Eq. 8）来解决。