Skip to main content
QUICK REVIEW

[论文解读] NetGPT: Generative Pretrained Transformer for Network Traffic

Xuying Meng, Chungang Lin|arXiv (Cornell University)|Apr 19, 2023
Internet Traffic Analysis and Secure E-votingComputer Science被引用 18
一句话总结

NetGPT 是首个用于网络流量的预训练模型,通过将多模式流量编码到统一的语义空间,并利用提示和头部字段洗牌来适应下游任务,既支持流量理解又支持生成。

ABSTRACT

All data on the Internet are transferred by network traffic, thus accurately modeling network traffic can help improve network services quality and protect data privacy. Pretrained models for network traffic can utilize large-scale raw data to learn the essential characteristics of network traffic, and generate distinguishable results for input traffic without considering specific downstream tasks. Effective pretrained models can significantly optimize the training efficiency and effectiveness of downstream tasks, such as application classification, attack detection and traffic generation. Despite the great success of pretraining in natural language processing, there is no work in the network field. Considering the diverse demands and characteristics of network traffic and network tasks, it is non-trivial to build a pretrained model for network traffic and we face various challenges, especially the heterogeneous headers and payloads in the multi-pattern network traffic and the different dependencies for contexts of diverse downstream network tasks. To tackle these challenges, in this paper, we make the first attempt to provide a generative pretrained model NetGPT for both traffic understanding and generation tasks. We propose the multi-pattern network traffic modeling to construct unified text inputs and support both traffic understanding and generation tasks. We further optimize the adaptation effect of the pretrained model to diversified tasks by shuffling header fields, segmenting packets in flows, and incorporating diverse task labels with prompts. With diverse traffic datasets from encrypted software, DNS, private industrial protocols and cryptocurrency mining, expensive experiments demonstrate the effectiveness of our NetGPT in a range of traffic understanding and generation tasks on traffic datasets, and outperform state-of-the-art baselines by a wide margin.

研究动机与目标

  • 激发对一个通用的网络流量预训练模型的需求,该模型能够处理多样模式和任务。
  • 开发通用的编码与预训练流程,在没有任务标签的情况下学习流量的内在特征。
  • 通过微调策略和提示实现对下游任务(分类、攻击检测、流量生成)的高效适配。

提出的方法

  • 通过将每个字节翻译为十六进制并使用具有类似 WordPiece 的词表的分词器,对多模式流量进行通用编码。
  • 以 GPT-2 为基础进行自回归预训练,以学习流量序列的下一个标记概率。
  • 预训练使用来自多样网络的未标记数据来学习统一的语义空间。
  • 微调通过头部字段洗牌、流内分组的分组,以及基于提示的任务标注来增强适应性。
  • 流量理解使用带提示的生成式表述,将分类任务统一为文本对文本的问题。
  • 流量生成通过对 real traffic 使用 Jensen-Shannon Divergence 评估保真度,并分析头部字段分布。

实验结果

研究问题

  • RQ1在没有任务特定标签的情况下,单个预训练模型是否能够捕捉到多样网络流量模式(明文和加密)的内在特征?
  • RQ2微调策略(头部字段洗牌、流级分组内的分组、提示)如何提升在流量理解和生成任务上的表现?
  • RQ3在实际数据集上,NetGPT 在分类/检测和流量生成任务上超越特定任务基线的程度如何?
  • RQ4所生成流量在代表头部字段分布方面的保真度与真实流量相比如何?

主要发现

  • NetGPT及其变体 NetGPT-A 在跨包级和流级设置下的流量理解任务上取得了强劲的性能。
  • NetGPT 胜过 GPT-2 基线,在对比 ET-BERT 时表现具有竞争力,尤其在包级任务和某些流级设置中收益显著。
  • 对头部字段进行洗牌并在流中对分组进行分段可提升下游性能,而过多的训练轮次可能导致过拟合。
  • 在流量生成方面,NetGPT 在若干数据集上产生的 Jensen-Shannon Divergence 低于 GPT-2,表明对真实流量分布的保真度更高。
  • NetGPT 对未见数据集(Cybermining)显示出强泛化能力,并在多样数据集(ISXW、DoHBrw、USTCTFC、PrivII)上保持鲁棒性。
  • 消融研究表明头部字段洗牌和分组分段对整体提升都很重要。

阅读本论文

在 Nubint 阅读器中打开原文,边读边就任意段落提问。

在 Nubint 中阅读本论文

本解读由 AI 根据论文原文生成,引用前请核对原文。