[Paper Review] User Clustering in Online Advertising via Topic Models

Sahin Cem Geyik, Ali Dasdan+1 more|arXiv (Cornell University)|Jan 26, 2015

Recommender Systems and Techniques16 references3 citations

TL;DR

This paper proposes a topic model-based user clustering method for online advertising that improves action prediction accuracy by grouping users with similar event histories into clusters. By modifying pLSA to assume conditional independence between user and cluster given event history, the method enables efficient, real-time cluster assignment for new users without retraining, achieving lower eCPA and eCPC in A/B testing compared to baseline methods.

ABSTRACT

In the domain of online advertising, our aim is to serve the best ad to a user who visits a certain webpage, to maximize the chance of a desired action to be performed by this user after seeing the ad. While it is possible to generate a different prediction model for each user to tell if he/she will act on a given ad, the prediction result typically will be quite unreliable with huge variance, since the desired actions are extremely sparse, and the set of users is huge (hundreds of millions) and extremely volatile, i.e., a lot of new users are introduced everyday, or are no longer valid. In this paper we aim to improve the accuracy in finding users who will perform the desired action, by assigning each user to a cluster, where the number of clusters is much smaller than the number of users (in the order of hundreds). Each user will fall into the same cluster with another user if their event history are similar. For this purpose, we modify the probabilistic latent semantic analysis (pLSA) model by assuming the independence of the user and the cluster id, given the history of events. This assumption helps us to identify a cluster of a new user without re-clustering all the users. We present the details of the algorithm we employed as well as the distributed implementation on Hadoop, and some initial results on the clusters that were generated by the algorithm.

Motivation & Objective

• To address the challenge of sparse and volatile user action data in online advertising by improving prediction accuracy for rare events.
• To enable efficient, scalable user clustering that supports real-time ad bidding with minimal retraining overhead.
• To develop a clustering method that assigns new users to clusters without recomputing the entire model from scratch.
• To enhance bid accuracy by using cluster-level action probabilities as features in real-time bidding systems.
• To evaluate the method's impact on key performance metrics like eCPA and eCPC in a real-world advertising environment.

Proposed method

• The method modifies probabilistic latent semantic analysis (pLSA) by assuming that user ID and cluster ID are conditionally independent given a user’s event history, enabling efficient inference.
• It uses beacon events (tracking pixels) as the primary signal for user behavior, treating each beacon as a 'word' in a user's behavioral 'document'.
• The algorithm applies the expectation-maximization (EM) framework to learn cluster assignments and probabilities, with a key optimization reducing EM computation cost.
• The system is implemented in Apache Pig on a Hadoop cluster to handle large-scale, distributed data processing across hundreds of millions of users.
• Cluster membership is determined at runtime by analyzing a new user’s beacon history, avoiding the need to store cluster assignments for all users.
• The cluster ID is used as a feature alongside campaign and publisher attributes to compute action probability estimates for bidding.

Experimental results

Research questions

• RQ1 Can user clustering based on event history improve action prediction accuracy in online advertising despite sparse and volatile user data?
• RQ2 How can a scalable clustering method be designed to assign new users to clusters without retraining the entire model?
• RQ3 Does using topic model-based clustering lead to better performance in real-time bidding systems compared to traditional similarity-based methods?
• RQ4 To what extent does cluster-based action probability estimation reduce eCPA and eCPC in real-world advertising campaigns?
• RQ5 How does the conditional independence assumption between user and cluster given event history improve model efficiency and inference speed?

Key findings

• The proposed topic model-based clustering method achieved significantly lower effective cost per action (eCPA) than the baseline cosine similarity method over a 10-day A/B test.
• The method also achieved lower effective cost per click (eCPC), indicating improved bid efficiency and better alignment with actual user actions.
• The performance improvement was consistent across the test period, with the new method outperforming the baseline in both eCPA and eCPC metrics.
• The results suggest that cluster-level action probability estimates are more accurate than per-user estimates due to reduced variance from data sparsity.
• The model's design allows real-time cluster assignment for new users without retraining, enabling scalable deployment in dynamic advertising environments.
• The observed trend of increasing eCPA toward the end of the test period was attributed to attribution latency in action tracking, not model failure.