[Paper Review] Augmentation Scheme for Dealing with Imbalanced Network Traffic Classification Using Deep Learning

Ramin Hasibi, Matin Shokri+1 more|arXiv (Cornell University)|Jan 1, 2019

Internet Traffic Analysis and Secure E-voting23 references33 citations

TL;DR

The paper presents an LSTM-based data augmentation scheme combined with KDE-based feature replication to balance imbalanced network traffic datasets and improve CRNN-based traffic classification performance.

ABSTRACT

One of the most important tasks in network management is identifying different types of traffic flows. As a result, a type of management service, called Network Traffic Classifier (NTC), has been introduced. One type of NTCs that has gained huge attention in recent years applies deep learning on packets in order to classify flows. Internet is an imbalanced environment i.e., some classes of applications are a lot more populated than others e.g., HTTP. Additionally, one of the challenges in deep learning methods is that they do not perform well in imbalanced environments in terms of evaluation metrics such as precision, recall, and $\mathrm{F_1}$ measure. In order to solve this problem, we recommend the use of augmentation methods to balance the dataset. In this paper, we propose a novel data augmentation approach based on the use of Long Short Term Memory (LSTM) networks for generating traffic flow patterns and Kernel Density Estimation (KDE) for replicating the numerical features of each class. First, we use the LSTM network in order to learn and generate the sequence of packets in a flow for classes with less population. Then, we complete the features of the sequence with generating random values based on the distribution of a certain feature, which will be estimated using KDE. Finally, we compare the training of a Convolutional Recurrent Neural Network (CRNN) in large-scale imbalanced, sampled, and augmented datasets. The contribution of our augmentation scheme is then evaluated on all of the datasets through measurements of precision, recall, and F1 measure for every class of application. The results demonstrate that our scheme is well suited for network traffic flow datasets and improves the performance of deep learning algorithms when it comes to above-mentioned metrics.

Motivation & Objective

• Address imbalanced class distributions in real-world network traffic datasets.
• Develop an augmentation scheme that preserves class semantics while expanding minority classes.
• Evaluate whether the augmented data improves deep learning classifier performance on traffic classification tasks.
• Compare augmentation with simple oversampling approaches on large-scale traffic data.

Proposed method

• Use LSTM networks to learn and generate packet-direction and TCP window-size sequences for minority classes.
• Estimate feature distributions with Kernel Density Estimation (KDE) for numerical features and sample from these PDFs to generate new flows.
• Assemble generated sequences and KDE-based features into augmented flow samples (up to 20 packets per flow, pad with zeros).
• Train a Convolutional Recurrent Neural Network (CRNN) on augmented data using a two-conv-layer, LSTM, and fully connected architecture with dropout, followed by softmax across 19 classes.
• Evaluate augmentation by comparing precision, recall, and F1 against a baseline and an oversampling approach on actual, sampled, and augmented datasets.

Experimental results

Research questions

• RQ1 Can LSTM-based sequence generation plus KDE-based feature replication mitigate class imbalance in network traffic datasets?
• RQ2 Does the augmentation improve per-class precision, recall, and F1 compared to oversampling?
• RQ3 How does CRNN performance on imbalanced traffic data change when trained on augmented versus non-augmented data?
• RQ4 How does augmentation affect overall accuracy and confusion among major and minor classes?

Key findings

• Augmentation improves recall across augmented classes compared to the actual and oversampled datasets.
• Overall F1 performance is better with augmentation than with simple sampling across all classes.
• CRNN trained on augmented data achieves higher accuracy and reduced false negatives, visible in the confusion matrix shift toward correct predictions.
• Accuracy increases by 6.56 percentage points when using the augmentation scheme versus the actual dataset.
• In some high-population classes, precision may slightly decrease, but recalls improve for minority classes, contributing to higher overall metrics.