[Paper Review] On generating network traffic datasets with synthetic attacks for intrusion detection

Carlos García Cordero, Emmanouil Vasilomanolakis+3 more|arXiv (Cornell University)|May 1, 2019

Network Security and Intrusion Detection4 citations

TL;DR

This paper introduces ID2T, an open-source toolkit that generates synthetic network attack traffic by injecting realistic, labeled attacks into real background network traffic, preserving the original traffic's statistical properties to ensure undetectability. The key contribution is a reproducible, privacy-preserving method for creating high-fidelity intrusion detection datasets without relying on outdated or private datasets.

ABSTRACT

Most research in the area of intrusion detection requires datasets to develop, evaluate or compare systems in one way or another. In this field, however, finding suitable datasets is a challenge on to itself. Most publicly available datasets have negative qualities that limit their usefulness. In this article, we propose ID2T (Intrusion Detection Dataset Toolkit) to tackle this problem. ID2T facilitates the creation of labeled datasets by injecting synthetic attacks into background traffic. The injected synthetic attacks blend themselves with the background traffic by mimicking the background traffic's properties to eliminate any trace of ID2T's usage. This work has three core contribution areas. First, we present a comprehensive survey on intrusion detection datasets. In the survey, we propose a classification to group the negative qualities we found in the datasets. Second, the architecture of ID2T is revised, improved and expanded. The architectural changes enable ID2T to inject recent and advanced attacks such as the widespread EternalBlue exploit or botnet communication patterns. The toolkit's new functionality provides a set of tests, known as TIDED (Testing Intrusion Detection Datasets), that help identify potential defects in the background traffic into which attacks are injected. Third, we illustrate how ID2T is used in different use-case scenarios to evaluate the performance of anomaly and signature-based intrusion detection systems in a reproducible manner. ID2T is open source software and is made available to the community to expand its arsenal of attacks and capabilities.

Motivation & Objective

• Address the longstanding challenge of unreliable, outdated, or incomplete publicly available intrusion detection datasets.
• Overcome limitations of existing datasets, such as outdated traffic characteristics, lack of labeled attacks, and privacy concerns.
• Provide a reproducible and extensible framework for generating labeled network traffic datasets with synthetic attacks.
• Enable researchers to evaluate intrusion detection systems (NIDS) on modern, realistic traffic without requiring access to sensitive or proprietary data.
• Support the creation of datasets that reflect current network behaviors and emerging threats, such as EternalBlue and botnet communication patterns.

Proposed method

• Inject synthetic attacks into real background network traffic using a toolkit that mimics the statistical properties of the original traffic to avoid detection.
• Revise and expand the ID2T architecture to support modern, advanced attacks, including the EternalBlue exploit and botnet communication patterns.
• Integrate a new testing module called TIDED to validate the quality and realism of background traffic before attack injection.
• Use TIDED to detect potential defects in background traffic, such as inconsistencies with expected network characteristics (e.g., backbone network behavior).
• Enable researchers to customize datasets by specifying background traffic and attack parameters, ensuring reproducibility across experiments.
• Support both anomaly-based and signature-based NIDS evaluation by generating attacks that are detectable using known signatures or behavioral patterns.

Experimental results

Research questions

• RQ1 How can synthetic network attack traffic be generated in a way that preserves the statistical properties of real background traffic to avoid detection?
• RQ2 What are the key shortcomings of existing public intrusion detection datasets, and how can they be systematically classified and addressed?
• RQ3 Can a toolkit like ID2T enable reproducible evaluation of NIDS across different time periods and network conditions without relying on static datasets?
• RQ4 To what extent can ID2T generate realistic, modern attacks such as EternalBlue or botnet C&C traffic that reflect current threat landscapes?
• RQ5 How can the interaction between attacks and background traffic be modeled to improve realism, especially in terms of network-level side effects like retransmissions or window size changes?

Key findings

• ID2T successfully generates synthetic attacks that blend seamlessly with background traffic by replicating its statistical properties, making the presence of the toolkit undetectable.
• The TIDED module effectively identifies defects in background traffic, such as deviations from expected backbone network behavior, improving dataset quality before attack injection.
• ID2T enables the reproduction of past NIDS evaluation results (e.g., RNN-based detection) on modern network traffic, demonstrating its utility for cross-temporal validation.
• Signature-based NIDS like Bro and Suricata successfully detect expected attacks injected via ID2T, confirming the tool’s effectiveness in validating deployed system behavior.
• ID2T supports the injection of advanced, contemporary attacks such as EternalBlue and botnet communication, expanding its relevance to current threat models.
• The toolkit provides a scalable, privacy-preserving alternative to static datasets, allowing researchers to generate custom, labeled datasets on demand without data sharing or anonymization issues.