[Paper Review] Self-supervised Auxiliary Learning with Meta-paths for Heterogeneous Graphs

Dasol Hwang, Jinyoung Park+4 more|arXiv (Cornell University)|Jul 16, 2020

Advanced Graph Neural NetworksComputer Science69 references38 citations

TL;DR

The paper introduces SELAR, a meta-learning framework that uses meta-path predictions as self-supervised auxiliary tasks on heterogeneous graphs, automatically balancing them to enhance primary tasks like link prediction and node classification, with an optional HintNet for harder tasks.

ABSTRACT

Graph neural networks have shown superior performance in a wide range of applications providing a powerful representation of graph-structured data. Recent works show that the representation can be further improved by auxiliary tasks. However, the auxiliary tasks for heterogeneous graphs, which contain rich semantic information with various types of nodes and edges, have less explored in the literature. In this paper, to learn graph neural networks on heterogeneous graphs we propose a novel self-supervised auxiliary learning method using meta-paths, which are composite relations of multiple edge types. Our proposed method is learning to learn a primary task by predicting meta-paths as auxiliary tasks. This can be viewed as a type of meta-learning. The proposed method can identify an effective combination of auxiliary tasks and automatically balance them to improve the primary task. Our methods can be applied to any graph neural networks in a plug-in manner without manual labeling or additional data. The experiments demonstrate that the proposed method consistently improves the performance of link prediction and node classification on heterogeneous graphs.

Motivation & Objective

• Leverage the rich semantic information in heterogeneous graphs through meta-path predictions as self-supervised auxiliary tasks.
• Automatically select and weight multiple auxiliary tasks to improve a primary task without extra data labels.
• Introduce a meta-learning-based balancing mechanism to mitigate negative transfer and task dominance.
• Enhance learning with HintNet to make challenging auxiliary tasks more solvable for the primary-task learner.

Proposed method

• Use meta-path predictions as auxiliary tasks to capture heterogeneous graph semantics.
• Employ a meta-learning-based weighting function to softly balance auxiliary tasks with the primary task.
• Introduce a nested optimization framework approximated by gradient-based updates to learn task weights (Theta) and model parameters (w).
• Implement a Hint Network (HintNet) to adjust auxiliary-task hints and improve learner performance.
• Provide cross-validation-style meta-dataset updates to mitigate meta-overfitting (3-fold CV used in experiments).
• Demonstrate plug-in applicability to existing GNNs (e.g., GCN, GAT, GIN, SGConv, GTN).

Experimental results

Research questions

• RQ1 Do meta-path prediction improve representation learning on heterogeneous graphs?
• RQ2 Can SELAR and HintNet further improve meta-path-based auxiliary learning for primary tasks?
• RQ3 Why are the proposed methods effective—what is the role of weighting and hard negatives in practice?
• RQ4 How does cross-validation influence meta-overfitting in meta-learning for auxiliary tasks?

Key findings

• Meta-path prediction consistently improves link prediction and node classification across multiple GNN backbones.
• SELAR and SELAR+Hint achieve notable absolute gains (e.g., up to about 2.0% on Last-FM and 2.7% on Book-Crossing for link prediction; up to about 3.0% on IMDB for node classification) compared to vanilla training.
• Merely adding meta-paths without meta-learning can slightly help or hurt; SELAR and HintNet provide the strongest improvements.
• The weighting function learned by SELAR+HintNet focuses on hard examples and effectively down-weights easy samples, akin to focal loss behavior.
• Cross-validation for meta-learning reduces meta-overfitting and yields larger gains than single-fold configurations.