[Paper Review] FedCTR: Federated Native Ad CTR Prediction with Multi-Platform User Behavior Data

Chuhan Wu, Fangzhao Wu+3 more|arXiv (Cornell University)|Jul 23, 2020

Recommender Systems and Techniques39 references5 citations

TL;DR

FedCTR proposes a privacy-preserving federated learning framework for native ad CTR prediction by jointly modeling user interests from multi-platform behavior data without centralized data storage. It uses local differential privacy (LDP) on platform-level embeddings and differential privacy (DP) on aggregated embeddings, achieving state-of-the-art performance even with limited in-platform behavior data while effectively protecting user privacy.

ABSTRACT

Native ad is a popular type of online advertisement which has similar forms with the native content displayed on websites. Native ad CTR prediction is useful for improving user experience and platform revenue. However, it is challenging due to the lack of explicit user intent, and users' behaviors on the platform with native ads may not be sufficient to infer their interest in ads. Fortunately, user behaviors exist on many online platforms and they can provide complementary information for user interest mining. Thus, leveraging multi-platform user behaviors is useful for native ad CTR prediction. However, user behaviors are highly privacy-sensitive and the behavior data on different platforms cannot be directly aggregated due to user privacy concerns and data protection regulations like GDPR. Existing CTR prediction methods usually require centralized storage of user behavior data for user modeling and cannot be directly applied to the CTR prediction task with multi-platform user behaviors. In this paper, we propose a federated native ad CTR prediction method named FedCTR, which can learn user interest representations from their behaviors on multiple platforms in a privacy-preserving way. On each platform a local user model is used to learn user embeddings from the local user behaviors on that platform. The local user embeddings from different platforms are uploaded to a server for aggregation, and the aggregated user embeddings are sent to the ad platform for CTR prediction. Besides, we apply LDP and DP techniques to the local and aggregated user embeddings respectively for better privacy protection. Moreover, we propose a federated framework for model training with distributed models and user behaviors. Extensive experiments on real-world dataset show that FedCTR can effectively leverage multi-platform user behaviors for native ad CTR prediction in a privacy-preserving manner.

Motivation & Objective

• To address the challenge of low CTR prediction accuracy in native ads due to lack of explicit user intent and insufficient in-platform behavior data.
• To enable effective user interest modeling from multi-platform user behaviors without centralized data collection, respecting privacy regulations like GDPR.
• To develop a federated learning framework that coordinates distributed user embeddings from multiple platforms while preserving user privacy at both local and aggregated levels.
• To evaluate the trade-off between CTR prediction performance and privacy protection using LDP and DP techniques.

Proposed method

• Each platform trains a local user model to learn user embeddings from its own behavior logs, preserving data locally.
• Local user embeddings are uploaded to a central user server for aggregation using an attention-based aggregator that models the informativeness of each platform’s contribution.
• Local embeddings are protected using Local Differential Privacy (LDP) to prevent inference of raw user behaviors from the uploaded embeddings.
• Aggregated embeddings are further protected using Differential Privacy (DP) to ensure privacy of the combined user representation.
• A federated training framework shares model gradients across platforms instead of raw behaviors, enabling collaborative model training without data leakage.
• The final CTR prediction is performed using a dot product interaction layer between the aggregated user embedding and ad features, which outperforms other scoring mechanisms.

Experimental results

Research questions

• RQ1 Can multi-platform user behavior data be effectively leveraged for native ad CTR prediction when in-platform behavior data is limited?
• RQ2 How can user privacy be preserved when aggregating user embeddings from multiple platforms with sensitive behavior logs?
• RQ3 What is the optimal trade-off between CTR prediction performance and privacy protection when applying LDP and DP to local and aggregated embeddings?
• RQ4 Which aggregation and scoring mechanisms (e.g., attention, dot product) yield the best performance in a federated CTR prediction setting?

Key findings

• The proposed FedCTR framework achieves superior CTR prediction performance by effectively combining user behavior data from multiple platforms, especially when in-platform behavior data is scarce.
• Using a dot product interaction layer between user and ad embeddings yields the best CTR prediction performance, outperforming FM, dense layers, and outer product mechanisms.
• An attention-based aggregator significantly improves performance over average pooling, max pooling, and concatenation by modeling the informativeness of each platform’s local embedding.
• Applying a moderate LDP parameter (λ_LDP = 0.01) and DP parameter (λ_DP = 0.005) achieves a strong balance between privacy protection and prediction accuracy.
• The aggregated user embedding is more difficult to reverse-engineer than local embeddings, indicating that aggregation enhances privacy by obfuscating platform-specific behavior patterns.