[Paper Review] JITuNE: Just-In-Time Hyperparameter Tuning for Network Embedding Algorithms

Mengying Guo, Tao Yi+2 more|arXiv (Cornell University)|Jan 16, 2021

Advanced Graph Neural Networks33 references4 citations

TL;DR

JITUne proposes a just-in-time hyperparameter tuning framework for network embedding algorithms that accelerates tuning via hierarchical network synopses and knowledge transfer, achieving state-of-the-art performance within constrained time budgets. It enables faster, more efficient optimization by tuning on progressively coarsened network representations and transferring learned configurations to the full network.

ABSTRACT

Network embedding (NE) can generate succinct node representations for massive-scale networks and enable direct applications of common machine learning methods to the network structure. Various NE algorithms have been proposed and used in a number of applications, such as node classification and link prediction. NE algorithms typically contain hyperparameters that are key to performance, but the hyperparameter tuning process can be time consuming. It is desirable to have the hyperparameters tuned within a specified length of time. Although AutoML methods have been applied to the hyperparameter tuning of NE algorithms, the problem of how to tune hyperparameters in a given period of time is not studied for NE algorithms before. In this paper, we propose JITuNE, a just-in-time hyperparameter tuning framework for NE algorithms. Our JITuNE framework enables the time-constrained hyperparameter tuning for NE algorithms by employing the tuning over hierarchical network synopses and transferring the knowledge obtained on synopses to the whole network. The hierarchical generation of synopsis and a time-constrained tuning method enable the constraining of overall tuning time. Extensive experiments demonstrate that JITuNE can significantly improve performances of NE algorithms, outperforming state-of-the-art methods within the same number of algorithm runs.

Motivation & Objective

• To address the prohibitively long runtime of hyperparameter tuning for network embedding (NE) algorithms on massive-scale networks.
• To develop a time-constrained optimization framework that guarantees a high-quality configuration within a user-specified time budget.
• To overcome limitations of existing methods, such as random sub-network sampling and inability to handle categorical hyperparameters.
• To enable effective knowledge transfer from small network synopses to the full network while preserving structural fidelity.
• To support diverse hyperparameter types, including categorical and numeric, for broader applicability.

Proposed method

• Constructs hierarchical network synopses through a coarsening process that preserves local and global network structures.
• Employs a similarity measurement module to identify the most representative synopsis for hyperparameter tuning based on structural similarity to the original network.
• Applies a time-constrained Bayesian optimization framework on the selected synopsis to efficiently explore the hyperparameter space.
• Transfers the learned optimal hyperparameter configuration from the synopsis to the full network using a meta-learner that accounts for structural and performance similarities.
• Integrates early stopping and progressive sampling to ensure the total tuning time remains within the user-defined limit.
• Supports both numeric and categorical hyperparameters, enabling broader applicability across diverse NE algorithms.

Experimental results

Research questions

• RQ1 Can a hierarchical synopsis-based approach reduce the time required for hyperparameter tuning in network embedding without sacrificing performance?
• RQ2 How effective is knowledge transfer from a network synopsis to the full network in improving tuning efficiency and accuracy?
• RQ3 Can the framework handle both numeric and categorical hyperparameters in a time-constrained setting?
• RQ4 How does JITuNE compare to state-of-the-art AutoML and NE-specific tuning methods in terms of performance and speed under time constraints?
• RQ5 To what extent does the choice of synopsis affect the quality of the final hyperparameter configuration?

Key findings

• JITUne significantly improves the performance of network embedding algorithms compared to state-of-the-art methods, achieving better results within the same number of algorithm runs.
• The framework reduces tuning time by leveraging hierarchical synopses and early stopping, ensuring results are delivered just in time.
• Stability analysis shows consistent hyperparameter configurations across five independent tuning runs on the Arxiv network, with performance variance below 0.001 in link prediction tasks.
• The method outperforms AutoNE and other AutoML baselines in both link prediction and node classification benchmarks across seven real-world networks.
• Knowledge transfer from synopses to the full network yields a performance gain of up to 5.2% in link prediction accuracy compared to tuning on the full network directly.
• The framework maintains high efficiency even on large-scale networks, with tuning time bounded by user-specified limits and no degradation in solution quality.