[Paper Review] Bridging the gap to real-world for network intrusion detection systems with data-centric approach

Gustavo de Carvalho Bertoli, Lourenço Alves Pereira+3 more|arXiv (Cornell University)|Oct 25, 2021

Network Security and Intrusion Detection17 references4 citations

TL;DR

This paper proposes a data-centric framework to generate up-to-date, labeled network intrusion detection (NIDS) datasets by combining real-time, publicly available MAWILab benign traffic with on-demand, virtualized attack traffic using Kali Linux containers. The approach produces a reproducible, labeled dataset of 835,941 packets with 54% attack samples (22 port scanning classes) and 46% benign traffic, addressing dataset aging and real-world applicability gaps in ML-based NIDS research.

ABSTRACT

Most research using machine learning (ML) for network intrusion detection systems (NIDS) uses well-established datasets such as KDD-CUP99, NSL-KDD, UNSW-NB15, and CICIDS-2017. In this context, the possibilities of machine learning techniques are explored, aiming for metrics improvements compared to the published baselines (model-centric approach). However, those datasets present some limitations as aging that make it unfeasible to transpose those ML-based solutions to real-world applications. This paper presents a systematic data-centric approach to address the current limitations of NIDS research, specifically the datasets. This approach generates NIDS datasets composed of the most recent network traffic and attacks, with the labeling process integrated by design.

Motivation & Objective

• Address the critical gap in NIDS research caused by outdated, aging datasets like KDD-CUP99 and CICIDS-2017, which limit real-world applicability.
• Overcome the limitations of model-centric approaches by shifting focus to continuous, data-driven dataset generation that reflects current network traffic and evolving attack patterns.
• Develop a reproducible, systematic framework for generating labeled NIDS datasets using real, up-to-date benign traffic and on-demand attack simulations.
• Enable continuous model retraining with current data to combat concept drift and improve generalization across diverse network architectures.
• Provide a publicly available, open-source pipeline for generating labeled NIDS datasets to support future research and real-world deployment.

Proposed method

• Use Docker containers with Kali Linux to generate on-demand, reproducible attack traffic against cloud-based target instances.
• Deploy UDP-based command-and-control mechanisms to trigger packet capture (using tcpdump) on target cloud instances, ensuring precise labeling via attacker IP filtering.
• Collect and merge network traffic files from multiple cloud instances, each labeled with attack type and timestamp, to form a unified attack dataset.
• Integrate up-to-date, publicly available MAWILab traffic (15-minute daily traces from trans-Pacific backbone) as the benign traffic source.
• Preprocess MAWILab data by filtering out packets labeled as anomalous or suspicious, then apply random sampling to balance the dataset and reduce class imbalance.
• Combine the processed benign and attack datasets into a single, tidy, labeled dataset with 41 features (packet-intrinsic and context-aware), ensuring stateless analysis compatibility.

Experimental results

Research questions

• RQ1 Can a data-centric approach generate up-to-date, labeled NIDS datasets that reflect current network traffic and evolving attack patterns, overcoming the limitations of aging benchmark datasets?
• RQ2 How can a systematic, reproducible pipeline be designed to generate labeled attack traffic and integrate it with real, benign network traffic for NIDS research?
• RQ3 To what extent can the proposed framework address concept drift in NIDS by enabling continuous dataset generation and model retraining?
• RQ4 How does the integration of real, up-to-date benign traffic (MAWILab) with synthetic attack traffic improve dataset realism and generalization potential?
• RQ5 What is the impact of controlled sampling and labeling on class distribution and model performance in imbalanced NIDS datasets?

Key findings

• The framework successfully generated a labeled NIDS dataset of 835,941 packets, with 54% classified as attack (22 distinct port scanning classes) and 46% as benign traffic.
• The attack dataset comprises 455,503 labeled samples from 22 different TCP port scanning techniques targeting 4 cloud-based instances.
• The benign dataset was derived from MAWILab traces collected on November 10 and November 29, 2020, after filtering out anomalous and suspicious traffic, resulting in 380,438 benign samples.
• The final dataset is balanced through random sampling, achieving a controlled 46% benign / 54% attack distribution to mitigate class imbalance.
• The dataset includes 41 features combining packet-intrinsic and context-aware attributes, suitable for stateless NIDS analysis.
• The entire pipeline, including attack generation and data collection, is reproducible and publicly available via the AB-TRAP GitHub repository.