[Paper Review] Ensemble Learning based Anomaly Detection for IoT Cybersecurity via Bayesian Hyperparameters Sensitivity Analysis

Tin Lai, Farnaz Farid+2 more|arXiv (Cornell University)|Jul 20, 2023

Anomaly Detection Techniques and ApplicationsComputer Science3 citations

TL;DR

This paper proposes a Bayesian hyperparameter-optimized ensemble learning framework for anomaly detection in IoT cybersecurity, leveraging diverse network features to improve detection accuracy. It demonstrates that tree-based models like XGBoost and GBM achieve superior performance through systematic hyperparameter sensitivity analysis, outperforming traditional methods on heterogeneous IoT datasets.

ABSTRACT

The Internet of Things (IoT) integrates more than billions of intelligent devices over the globe with the capability of communicating with other connected devices with little to no human intervention. IoT enables data aggregation and analysis on a large scale to improve life quality in many domains. In particular, data collected by IoT contain a tremendous amount of information for anomaly detection. The heterogeneous nature of IoT is both a challenge and an opportunity for cybersecurity. Traditional approaches in cybersecurity monitoring often require different kinds of data pre-processing and handling for various data types, which might be problematic for datasets that contain heterogeneous features. However, heterogeneous types of network devices can often capture a more diverse set of signals than a single type of device readings, which is particularly useful for anomaly detection. In this paper, we present a comprehensive study on using ensemble machine learning methods for enhancing IoT cybersecurity via anomaly detection. Rather than using one single machine learning model, ensemble learning combines the predictive power from multiple models, enhancing their predictive accuracy in heterogeneous datasets rather than using one single machine learning model. We propose a unified framework with ensemble learning that utilises Bayesian hyperparameter optimisation to adapt to a network environment that contains multiple IoT sensor readings. Experimentally, we illustrate their high predictive power when compared to traditional methods.

Motivation & Objective

• To address the challenge of detecting diverse cyberattacks in heterogeneous IoT environments with varying data types and network features.
• To improve anomaly detection performance by combining multiple machine learning models via ensemble learning.
• To identify the most influential hyperparameters in ensemble models using Bayesian optimization and sensitivity analysis.
• To evaluate model robustness across multiple IoT cybersecurity datasets, including DDoS, port scanning, and malware-based attacks.
• To provide actionable insights for IoT device designers and security analysts by identifying key network features indicative of specific attack types.

Proposed method

• The framework employs ensemble learning by combining multiple base estimators (e.g., XGBoost, LightGBM, Random Forest) to enhance predictive performance on heterogeneous IoT data.
• Bayesian hyperparameter optimization is used to automatically tune model configurations, focusing on learning rate, max depth, and subsample ratios.
• Sensitivity analysis is applied to assess the impact of individual hyperparameters on model performance, identifying those with the highest influence.
• Network-level features—such as packet counts, flag usage (SYN/ACK), and byte sizes in forward/backward directions—are extracted to represent attack behaviors.
• The model is trained and evaluated on diverse IoT anomaly detection datasets, including Mirai-based DDoS attacks and man-in-the-middle scenarios.
• A unified pipeline integrates data preprocessing, hyperparameter tuning, model training, and performance evaluation to ensure robustness across attack types.

Experimental results

Research questions

• RQ1 Which ensemble machine learning models exhibit the highest accuracy and recall in detecting IoT-based cyberattacks across heterogeneous datasets?
• RQ2 How do different hyperparameters influence the performance of ensemble models in IoT anomaly detection?
• RQ3 Which network features are most discriminative for identifying specific attack types such as Mirai DDoS, port scanning, or MITM attacks?
• RQ4 To what extent does Bayesian hyperparameter optimization improve model generalization and robustness compared to default configurations?
• RQ5 What are the key hyperparameters that significantly affect model performance, enabling targeted tuning for optimal results?

Key findings

• Tree-based boosting models such as XGBoost and Gradient Boosting Machines (GBM) consistently achieve the highest accuracy and recall across diverse IoT anomaly detection datasets.
• The sensitivity analysis reveals that only a small subset of hyperparameters—particularly learning rate, max depth, and subsample ratio—exert a dominant influence on model performance.
• Bayesian hyperparameter optimization significantly improves model generalization, outperforming standard grid or random search in tuning efficiency and final accuracy.
• Network features such as packet counts with specific TCP flags (e.g., SYN/ACK), byte sizes in forward/backward directions, and flow rates are highly discriminative for identifying attack types like DDoS, port scanning, and brute force attacks.
• The framework successfully detects complex attack patterns, including distributed Mirai botnet activities, by leveraging time-series and flow-based network features.
• The study provides actionable feature sets for security experts and device designers to build more resilient IoT monitoring systems based on identifiable attack signatures.