[Paper Review] Machine Learning based Anomaly Detection for 5G Networks

Jordan Lam, Hassan Abbas|arXiv (Cornell University)|Jan 1, 2020

Network Security and Intrusion DetectionComputer Science11 references23 citations

TL;DR

This paper proposes a Software-Defined Security (SDS) framework using a Convolutional Neural Network (CNN) optimized via Neural Architecture Search (NAS) to detect anomalous network traffic in 5G networks. By converting network flow data into image-like representations, the model achieves 100% accuracy in identifying benign traffic and 96.4% detection rate for anomalies, demonstrating strong potential for automated, scalable intrusion detection in complex 5G environments.

ABSTRACT

Protecting the networks of tomorrow is set to be a challenging domain due to increasing cyber security threats and widening attack surfaces created by the Internet of Things (IoT), increased network heterogeneity, increased use of virtualisation technologies and distributed architectures. This paper proposes SDS (Software Defined Security) as a means to provide an automated, flexible and scalable network defence system. SDS will harness current advances in machine learning to design a CNN (Convolutional Neural Network) using NAS (Neural Architecture Search) to detect anomalous network traffic. SDS can be applied to an intrusion detection system to create a more proactive and end-to-end defence for a 5G network. To test this assumption, normal and anomalous network flows from a simulated environment have been collected and analyzed with a CNN. The results from this method are promising as the model has identified benign traffic with a 100% accuracy rate and anomalous traffic with a 96.4% detection rate. This demonstrates the effectiveness of network flow analysis for a variety of common malicious attacks and also provides a viable option for detection of encrypted malicious network traffic.

Motivation & Objective

• To address the growing complexity and attack surface of 5G networks due to IoT proliferation, virtualization, and network slicing.
• To develop an automated, flexible, and scalable intrusion detection system using machine learning for end-to-end 5G network protection.
• To evaluate the effectiveness of converting network flow data into image representations for CNN-based anomaly detection.
• To demonstrate the feasibility of using automated machine learning (autoML) and NAS to design high-performing, domain-specific deep learning models for network security.

Proposed method

• Network flow data from a simulated 5G environment was collected, including both benign and anomalous traffic patterns.
• The data was pre-processed and transformed into image-like representations suitable for input into a CNN.
• A CNN architecture was automatically generated using Neural Architecture Search (NAS) to optimize model performance on the specific dataset.
• The model was trained using supervised learning on labeled network flow images, with performance evaluated using precision, recall, and F1 score metrics.
• The final model was evaluated on a test set to assess detection accuracy and robustness across different attack types.
• The system was designed to support real-time deployment by enabling dynamic policy updates based on detected threats.

Experimental results

Research questions

• RQ1 Can a CNN trained on image-converted network flow data achieve high detection accuracy for anomalous traffic in 5G networks?
• RQ2 How effective is NAS in generating an optimal CNN architecture for 5G network anomaly detection without manual design?
• RQ3 To what extent can machine learning-based models detect encrypted malicious traffic through flow-level analysis?
• RQ4 Can the proposed SDS framework provide scalable and adaptive security across diverse 5G network slices and heterogeneous architectures?
• RQ5 How does the model balance precision and recall in real-world scenarios with high volumes of benign and malicious traffic?

Key findings

• The model achieved 100% recall for benign traffic, correctly identifying all normal flows without false positives.
• The model detected 96.4% of anomalous traffic, with a precision of 100% for anomaly predictions.
• The F1 score for anomaly detection was 0.982, indicating a strong balance between precision and recall.
• The confusion matrix confirmed that only 3.6% of anomaly flows were misclassified as benign, while no benign flows were incorrectly flagged as anomalous.
• The NAS-optimized CNN outperformed baseline architectures, demonstrating the value of automated neural architecture search in security applications.
• The results suggest that image-based representation of network flows is a viable and effective approach for deep learning-based anomaly detection in 5G networks.