[Paper Review] GraphSW: a training protocol based on stage-wise training for GNN-based Recommender Model

Chang-You Tai, Meng-Ru Wu+2 more|arXiv (Cornell University)|Aug 13, 2019

Advanced Graph Neural Networks4 citations

TL;DR

GraphSW is a stage-wise training protocol that improves GNN-based recommender models by progressively incorporating knowledge graph (KG) information through iterative, incremental training. By training on subsets of KG entities in stages and transferring learned embeddings, GraphSW enhances model performance and enables convergence in high-order GNNs like KGCN, especially with small neighbor sampling sizes and increasing hop counts.

ABSTRACT

Recently, researchers utilize Knowledge Graph (KG) as side information in recommendation system to address cold start and sparsity issue and improve the recommendation performance. Existing KG-aware recommendation model use the feature of neighboring entities and structural information to update the embedding of currently located entity. Although the fruitful information is beneficial to the following task, the cost of exploring the entire graph is massive and impractical. In order to reduce the computational cost and maintain the pattern of extracting features, KG-aware recommendation model usually utilize fixed-size and random set of neighbors rather than complete information in KG. Nonetheless, there are two critical issues in these approaches: First of all, fixed-size and randomly selected neighbors restrict the view of graph. In addition, as the order of graph feature increases, the growth of parameter dimensionality of the model may lead the training process hard to converge. To solve the aforementioned limitations, we propose GraphSW, a strategy based on stage-wise training framework which would only access to a subset of the entities in KG in every stage. During the following stages, the learned embedding from previous stages is provided to the network in the next stage and the model can learn the information gradually from the KG. We apply stage-wise training on two SOTA recommendation models, RippleNet and Knowledge Graph Convolutional Networks (KGCN). Moreover, we evaluate the performance on six real world datasets, Last.FM 2011, Book-Crossing,movie, LFM-1b 2015, Amazon-book and Yelp 2018. The result of our experiments shows that proposed strategy can help both models to collect more information from the KG and improve the performance. Furthermore, it is observed that GraphSW can assist KGCN to converge effectively in high-order graph feature.

Motivation & Objective

• Address the challenge of computational cost and model non-convergence in high-order GNN-based recommendation models.
• Overcome limitations of fixed-size, random neighbor sampling in KG-aware models that restrict information access and hinder training stability.
• Improve performance and convergence of KGCN and RippleNet by systematically incorporating KG information in stages.
• Investigate whether resampling strategies and staged training can enhance model robustness and scalability across diverse datasets.
• Enable training of deeper GNNs with higher-order features by mitigating parameter explosion and noise accumulation.

Proposed method

• Introduce a stage-wise training framework where each stage processes a subset of entities from the knowledge graph.
• Transfer learned entity embeddings from previous stages to the next, enabling gradual information accumulation.
• Use fixed-size neighbor sampling per stage, but vary the sampled neighbors across stages to explore diverse graph structures.
• Apply the protocol to KGCN and RippleNet, training on different hop counts and neighbor sampling sizes.
• Transfer only KG representations across stages (not full model parameters) to preserve training stability and performance.
• Optimize hyperparameters such as neighbor sampling size and hop count to evaluate performance gains across datasets.

Experimental results

Research questions

• RQ1 Can a stage-wise training protocol improve the performance of GNN-based recommender models when using limited KG information?
• RQ2 Does progressive information aggregation through staged training help models like KGCN and RippleNet achieve better convergence, especially at higher hop counts?
• RQ3 How does neighbor sampling size affect model performance when using GraphSW compared to standard fixed-sampling strategies?
• RQ4 Can GraphSW mitigate the non-convergence issues in KGCN caused by increasing parameter dimensionality and noise in high-order graph features?
• RQ5 Does transferring only KG embeddings across stages preserve model performance better than transferring full model parameters?

Key findings

• GraphSW significantly improves AUC performance across all six datasets for both KGCN and RippleNet, with average improvements of up to 34.8% on Last.FM 2011 at hop 4.
• KGCN achieves its best performance with small neighbor sampling sizes (e.g., 16) when using GraphSW, indicating better feature learning with limited but progressive neighbor access.
• For KGCN, GraphSW enables stable training and performance gains even at hop 4, where standard training fails to converge due to parameter explosion and noise.
• On Yelp 2018, GraphSW improves KGCN’s AUC from 0.8720 (standard) to 0.9038 (GraphSW) at hop 4, demonstrating effectiveness in large-scale KGs.
• Transferring only KG representations across stages yields better performance than transferring full model parameters, confirming the importance of preserving learned KG embeddings.
• RippleNet benefits most from GraphSW with larger neighbor sampling sizes, while KGCN performs best with smaller samples, highlighting architectural differences in feature aggregation.