[Paper Review] Click-Through Rate Prediction Using Graph Neural Networks and Online Learning

Farzaneh Rajabi, Jack Siyuan He|arXiv (Cornell University)|May 9, 2021

Image and Video Quality Assessment18 references4 citations

TL;DR

This paper proposes a graph neural network (GNN)-based click-through rate (CTR) prediction model enhanced with online learning to capture dynamic user-item interactions. By modeling feature interactions via a graph structure and adapting in real time, the method achieves a test AUC of 0.7585 on a sub-sampled Criteo dataset, outperforming offline GNN and DeepFM baselines.

ABSTRACT

Recommendation systems have been extensively studied by many literature in the past and are ubiquitous in online advertisement, shopping industry/e-commerce, query suggestions in search engines, and friend recommendation in social networks. Moreover, restaurant/music/product/movie/news/app recommendations are only a few of the applications of a recommender system. A small percent improvement on the CTR prediction accuracy has been mentioned to add millions of dollars of revenue to the advertisement industry. Click-Through-Rate (CTR) prediction is a special version of recommender system in which the goal is predicting whether or not a user is going to click on a recommended item. A content-based recommendation approach takes into account the past history of the user's behavior, i.e. the recommended products and the users reaction to them. So, a personalized model that recommends the right item to the right user at the right time is the key to building such a model. On the other hand, the so-called collaborative filtering approach incorporates the click history of the users who are very similar to a particular user, thereby helping the recommender to come up with a more confident prediction for that particular user by leveraging the wider knowledge of users who share their taste in a connected network of users. In this project, we are interested in building a CTR predictor using Graph Neural Networks complemented by an online learning algorithm that models such dynamic interactions. By framing the problem as a binary classification task, we have evaluated this system both on the offline models (GNN, Deep Factorization Machines) with test-AUC of 0.7417 and on the online learning model with test-AUC of 0.7585 using a sub-sampled version of Criteo public dataset consisting of 10,000 data points.

Motivation & Objective

• Address the gap between offline evaluation metrics and online performance in CTR prediction by modeling dynamic user-recommender interactions.
• Improve CTR prediction accuracy by explicitly modeling high-order feature interactions using a graph neural network.
• Integrate online learning to enable real-time adaptation of the model to evolving user behavior.
• Overcome limitations of static offline models and black-box deep learning by combining explainable factorization techniques with dynamic learning.
• Develop a scalable and adaptive recommendation system that balances accuracy, explainability, and responsiveness to user feedback.

Proposed method

• Represent multi-field categorical features as nodes in a heterogeneous graph, where edges encode feature interactions.
• Apply a Graph Interaction Network (GIN) variant to aggregate neighborhood information and update node representations through message passing.
• Integrate online learning via stochastic gradient updates using a sliding window of recent user interactions to adapt model parameters in real time.
• Combine the GNN’s ability to model high-order feature interactions with a lightweight online update rule to maintain model relevance.
• Use a binary cross-entropy loss function for end-to-end training of the GNN, optimized via Adam with learning rate scheduling.
• Apply threshold-based inference (e.g., 0.3) to convert predicted click probabilities into top-K recommendations.

Experimental results

Research questions

• RQ1 Can a GNN-based CTR model with online fine-tuning outperform static offline models in real-world dynamic environments?
• RQ2 How do high-order feature interactions modeled via graph structures improve CTR prediction beyond first- and second-order methods?
• RQ3 To what extent does online learning enhance model generalization and alignment with online A/B testing outcomes?
• RQ4 What is the impact of data imbalance and class skew on model performance, and how can it be mitigated in CTR prediction?
• RQ5 Can a hybrid model combining GNNs and online learning achieve better AUC and recall than baseline models like DeepFM and GNNs alone?

Key findings

• The online learning-enhanced GNN achieved a test AUC of 0.7585, surpassing the offline GNN (0.7417) and DeepFM baselines.
• The model demonstrated improved robustness to concept drift by adapting to new user behavior patterns through continuous online updates.
• Despite high class imbalance (negative class accuracy: 0.96), the model maintained a relatively high F1 score at optimal threshold (0.3), indicating effective handling of skewed data.
• The confusion matrix revealed a high false negative rate, confirming the persistent challenge of data imbalance, which could be mitigated with resampling or class weighting.
• The model’s recall was significantly lower than the baseline, suggesting that future work should prioritize improving recall through data rebalancing and threshold tuning.
• Ablative ablation of GNN components (e.g., edge-wise attention, residual connections) could yield further AUC improvements, though this was left for future work.