[Paper Review] How to Achieve High Classification Accuracy with Just a Few Labels: A Semi-supervised Approach Using Sampled Packets

Shahbaz Rezaei, Xin Liu|arXiv (Cornell University)|Dec 23, 2018

Internet Traffic Analysis and Secure E-voting82 citations

TL;DR

A semi-supervised traffic classifier pretrained on large unlabeled data learns from sampled packets and achieves near-supervised accuracy with only 20 labeled flows per class, including QUIC traffic.

ABSTRACT

Network traffic classification, which has numerous applications from security to billing and network provisioning, has become a cornerstone of today's computer networks. Previous studies have developed traffic classification techniques using classical machine learning algorithms and deep learning methods when large quantities of labeled data are available. However, capturing large labeled datasets is a cumbersome and time-consuming process. In this paper, we propose a semi-supervised approach that obviates the need for large labeled datasets. We first pre-train a model on a large unlabeled dataset where the input is the time series features of a few sampled packets. Then the learned weights are transferred to a new model that is re-trained on a small labeled dataset. We show that our semi-supervised approach achieves almost the same accuracy as a fully-supervised method with a large labeled dataset, though we use only 20 samples per class. In tests based on a dataset generated from the more challenging QUIC protocol, our approach yields 98% accuracy. To show its efficacy, we also test our approach on two public datasets. Moreover, we study three different sampling techniques and demonstrate that sampling packets from an arbitrary portion of a flow is sufficient for classification.

Motivation & Objective

• Motivate reducing labeled data needs for network traffic classification.
• Propose a semi-supervised pipeline combining unlabeled pre-training and small labeled re-training.
• Explore packet sampling strategies to enable efficient time-series based classification.
• Demonstrate generalizability across QUIC and public datasets.

Proposed method

• Pre-train a CNN on a large unlabeled dataset to predict statistical flow features from time-series samples.
• Transfer learned weights to a second model and re-train with a small labeled dataset.
• Use time-series features from sampled packets as input and predict statistical features as targets.
• Employ three sampling methods: fixed step, random, and incremental sampling.
• Evaluate with QUIC traffic and public datasets, comparing against fully supervised baselines.

Experimental results

Research questions

• RQ1 Can a model pre-trained on unlabeled data reduce the amount of labeled data needed for accurate traffic classification?
• RQ2 How do different packet sampling strategies affect classification performance with limited labels?
• RQ3 Does transfer learning from unlabeled data generalize across datasets and protocols (e.g., QUIC)?

Key findings

• The semi-supervised approach achieves near fully-supervised accuracy with only 20 labeled flows per class on QUIC-derived data (incremental sampling: 98.53% vs upper-bound 98.99%).
• Incremental sampling generally outperforms fixed step and random sampling in accuracy gains.
• A pre-trained model improves accuracy by about 10% over non-pre-trained baselines under certain sampling settings.
• Training with unlabeled Waikato data for pre-training still yields sizable gains when target traffic is absent in pre-training (up to ~10% improvement).
• On Ariel public data, pre-training plus small labeled re-training reaches near upper-bound performance with relatively few labeled flows (e.g., around 30 labeled flows per class to reach upper-bound).
• A random-forest baseline using statistical features achieves 99.87% on QUIC data, indicating sampling degrades performance modestly compared to full-feature models.