[Paper Review] LCCDE: A Decision-Based Ensemble Framework for Intrusion Detection in The Internet of Vehicles

Li Yang, Abdallah Shami+2 more|arXiv (Cornell University)|Aug 5, 2022

Network Security and Intrusion Detection4 citations

TL;DR

This paper proposes LCCDE, a decision-based ensemble framework that selects the best-performing gradient-boosting model (XGBoost, LightGBM, or CatBoost) for each attack class based on prediction confidence, significantly improving intrusion detection accuracy in Internet of Vehicles (IoV) networks. The method achieves near-perfect F1-scores of 99.9997% on the Car-Hacking dataset and 99.811% on the CICIDS2017 dataset, outperforming state-of-the-art methods.

ABSTRACT

Modern vehicles, including autonomous vehicles and connected vehicles, have adopted an increasing variety of functionalities through connections and communications with other vehicles, smart devices, and infrastructures. However, the growing connectivity of the Internet of Vehicles (IoV) also increases the vulnerabilities to network attacks. To protect IoV systems against cyber threats, Intrusion Detection Systems (IDSs) that can identify malicious cyber-attacks have been developed using Machine Learning (ML) approaches. To accurately detect various types of attacks in IoV networks, we propose a novel ensemble IDS framework named Leader Class and Confidence Decision Ensemble (LCCDE). It is constructed by determining the best-performing ML model among three advanced ML algorithms (XGBoost, LightGBM, and CatBoost) for every class or type of attack. The class leader models with their prediction confidence values are then utilized to make accurate decisions regarding the detection of various types of cyber-attacks. Experiments on two public IoV security datasets (Car-Hacking and CICIDS2017 datasets) demonstrate the effectiveness of the proposed LCCDE for intrusion detection on both intra-vehicle and external networks.

Motivation & Objective

• To address the challenge of inconsistent machine learning model performance across different types of cyber-attacks in IoV networks.
• To improve intrusion detection accuracy by dynamically selecting the best-performing model for each attack class.
• To leverage prediction confidence scores to enhance decision-making in ensemble-based intrusion detection.
• To evaluate the framework on real-world IoV security datasets representing both intra-vehicle (CAN bus) and external network threats.
• To demonstrate superior performance compared to existing state-of-the-art machine learning and ensemble IDS approaches.

Proposed method

• The LCCDE framework selects the top-performing machine learning model (XGBoost, LightGBM, or CatBoost) for each attack class based on validation performance.
• For each attack type, the model with the highest prediction confidence is designated as the 'class leader' to make final decisions.
• The ensemble combines class-specific leader models, using their confidence scores to determine final intrusion classifications.
• The framework is trained and evaluated on two public IoV datasets: Car-Hacking (intra-vehicle network) and CICIDS2017 (external network).
• The method employs parallel execution and GPU support to maintain efficient inference times despite ensemble complexity.
• Prediction confidence is used as a decision criterion to improve robustness and accuracy in multi-class intrusion detection.

Experimental results

Research questions

• RQ1 Can a dynamic ensemble strategy that selects the best-performing model per attack class improve overall intrusion detection accuracy in IoV?
• RQ2 How does incorporating prediction confidence enhance decision-making in ensemble-based IDS for IoV?
• RQ3 Does the LCCDE framework outperform individual models and existing ensemble methods on standard IoV security benchmarks?
• RQ4 What is the trade-off between detection accuracy and execution time in the proposed LCCDE framework?
• RQ5 How does the framework perform across diverse attack types, including DoS, spoofing, and brute-force attacks, on both intra-vehicle and external network data?

Key findings

• The LCCDE framework achieved a near-perfect F1-score of 99.9997% on the Car-Hacking dataset, outperforming all compared models.
• On the CICIDS2017 dataset, LCCDE achieved an F1-score of 99.811%, representing a 0.11% improvement over the next best method.
• For every individual attack class, LCCDE achieved the highest F1-score, demonstrating consistent superiority across all threat types.
• The framework maintained a reasonable execution time of 185.1 seconds on Car-Hacking and 168.9 seconds on CICIDS2017, outperforming KNN and SVM while remaining efficient.
• The use of class leader models with confidence scoring significantly enhanced detection accuracy compared to static ensemble or single-model approaches.
• LCCDE outperformed state-of-the-art methods such as DCNN, LSTM-AE, and stacking ensembles in both F1-score and robustness across attack types.