[论文解读] DADNN: Multi-Scene CTR Prediction via Domain-Aware Deep Neural Network

Junyou He, Guibao Mei+4 more|arXiv (Cornell University)|Nov 24, 2020

Image and Video Quality Assessment参考文献 16被引用 6

一句话总结

本文提出DADNN，一种用于多场景点击率（CTR）预测的领域感知深度神经网络，通过共享底部模块进行通用表征学习，并利用领域特定的输出头捕捉场景特异性特征。通过整合知识迁移与MMoE以实现共享/判别性特征学习，DADNN在在线A/B测试中相较经过充分调优的DCN模型，点击率最高提升6.7%，每千次展示成本（CPM）最高提升3.0%，同时通过单一模型架构支持多个场景，显著降低训练与推理成本。

ABSTRACT

Click through rate(CTR) prediction is a core task in advertising systems. The booming e-commerce business in our company, results in a growing number of scenes. Most of them are so-called long-tail scenes, which means that the traffic of a single scene is limited, but the overall traffic is considerable. Typical studies mainly focus on serving a single scene with a well designed model. However, this method brings excessive resource consumption both on offline training and online serving. Besides, simply training a single model with data from multiple scenes ignores the characteristics of their own. To address these challenges, we propose a novel but practical model named Domain-Aware Deep Neural Network(DADNN) by serving multiple scenes with only one model. Specifically, shared bottom block among all scenes is applied to learn a common representation, while domain-specific heads maintain the characteristics of every scene. Besides, knowledge transfer is introduced to enhance the opportunity of knowledge sharing among different scenes. In this paper, we study two instances of DADNN where its shared bottom block is multilayer perceptron(MLP) and Multi-gate Mixture-of-Experts(MMoE) respectively, for which we denote as DADNN-MLP and DADNN-MMoE.Comprehensive offline experiments on a real production dataset from our company show that DADNN outperforms several state-of-the-art methods for multi-scene CTR prediction. Extensive online A/B tests reveal that DADNN-MLP contributes up to 6.7% CTR and 3.0% CPM(Cost Per Mille) promotion compared with a well-engineered DCN model. Furthermore, DADNN-MMoE outperforms DADNN-MLP with a relative improvement of 2.2% and 2.7% on CTR and CPM respectively. More importantly, DADNN utilizes a single model for multiple scenes which saves a lot of offline training and online serving resources.

研究动机与目标

• 为解决因资源成本高与数据稀缺，对数百个低流量、长尾电商广告场景分别训练模型所带来的挑战。
• 通过在统一模型框架内同时学习共享表征与场景特异性表征，缓解多场景CTR预测中的领域偏移问题。
• 通过场景间知识迁移，提升数据稀缺场景下的模型泛化能力与性能，且无需外部教师网络。
• 实现单一模型在多个场景中的可扩展、资源高效部署，支持新场景的持续接入。
• 探索MMoE在CTR预测中显式建模场景间共性与差异性的有效性。

提出的方法

• DADNN在所有场景中使用共享底部模块，从多个场景的联合数据中学习通用且可迁移的表征。
• 每个场景配备一个领域特定的输出头，学习判别性特征，减少领域偏移，实现场景感知的预测。
• 通过损失函数实现知识迁移，促进场景间知识共享，尤其使低数据场景受益。
• 模型支持两种变体：DADNN-MLP采用多层感知机共享模块，DADNN-MMoE采用多门控专家模型（Multi-gate Mixture-of-Experts）以提升表征学习能力。
• MMoE通过动态分配门控权重给专家，使模型在参数增加极少的情况下，学习共享与场景特异性特征。
• 使用来自大规模场景的预训练嵌入层初始化模型参数，提升收敛速度与性能。

实验结果

研究问题

• RQ1 是否可通过单一深度学习模型有效服务多个低流量、长尾广告场景，而无需为每个场景单独训练模型？
• RQ2 在保持场景特异性特征的同时，如何缓解CTR预测中不同场景间的领域偏移？
• RQ3 场景间知识迁移在多大程度上提升性能，尤其是在数据稀缺场景中？
• RQ4 MMoE模块是否通过显式建模共享与独特表征，提升模型性能？
• RQ5 统一模型架构是否能高效扩展以支持新场景，且仅需极少再训练或重新配置？

主要发现

• 在在线A/B测试中，DADNN-MLP相较经过精心调优的DCN模型，点击率最高提升6.7%，每千次展示成本（CPM）最高提升3.0%。
• DADNN-MMoE相较DADNN-MLP在点击率上提升2.2%，在CPM上提升2.7%，证明显式建模共享与场景特异性特征的优势。
• 知识迁移通过增强场景间知识共享，显著提升模型性能，尤其在低数据场景中效果明显。
• 消融实验证实，将知识迁移与MMoE模块结合可获得最佳性能，相较基线模型绝对GAUC提升0.635%。
• MMoE模块在专家数量超过两个后收益递减，表明复杂度与性能之间存在最优平衡。
• 模型在使用单一模型支持所有场景的前提下，实现了显著的性能提升，同时大幅降低离线训练与在线推理成本。