[Paper Review] Hybrid Deep Learning Model using SPCAGAN Augmentation for Insider Threat Analysis

R G Gayathri, Atul Sajjanhar+1 more|arXiv (Cornell University)|Mar 6, 2022

Network Security and Intrusion Detection4 citations

TL;DR

This paper proposes SPCAGAN, a hybrid deep learning model that uses linear manifold learning-based GANs with a novel loss function to generate high-quality synthetic insider threat data, addressing class imbalance and improving anomaly detection. The approach achieves superior performance over state-of-the-art methods, with SPCAGAN reducing mode collapse and improving detection accuracy on benchmark datasets.

ABSTRACT

Cyberattacks from within an organization's trusted entities are known as insider threats. Anomaly detection using deep learning requires comprehensive data, but insider threat data is not readily available due to confidentiality concerns of organizations. Therefore, there arises demand to generate synthetic data to explore enhanced approaches for threat analysis. We propose a linear manifold learning-based generative adversarial network, SPCAGAN, that takes input from heterogeneous data sources and adds a novel loss function to train the generator to produce high-quality data that closely resembles the original data distribution. Furthermore, we introduce a deep learning-based hybrid model for insider threat analysis. We provide extensive experiments for data synthesis, anomaly detection, adversarial robustness, and synthetic data quality analysis using benchmark datasets. In this context, empirical comparisons show that GAN-based oversampling is competitive with numerous typical oversampling regimes. For synthetic data generation, our SPCAGAN model overcame the problem of mode collapse and converged faster than previous GAN models. Results demonstrate that our proposed approach has a lower error, is more accurate, and generates substantially superior synthetic insider threat data than previous models.

Motivation & Objective

• Address the challenge of class imbalance in insider threat detection due to limited real-world data.
• Develop a data augmentation method that generates diverse, high-fidelity synthetic insider threat instances to improve model generalization.
• Enhance adversarial robustness and detection performance using a hybrid Bayesian neural network (BNN) and GAN-based training.
• Overcome limitations of existing GANs, such as mode collapse and slow convergence, in insider threat data synthesis.
• Provide a systematic evaluation of synthetic data quality and anomaly detection performance on benchmark datasets.

Proposed method

• Propose SPCAGAN, a modified Auxiliary Classifier GAN (ACGAN) with linear manifold learning-based regularization to improve generator training.
• Integrate a novel loss function that encourages the generator to learn the intrinsic data manifold, enhancing data diversity and distribution fidelity.
• Use linear intrinsic dimension estimation to analyze and guide the generator in learning the true data distribution from heterogeneous sources.
• Apply adversarial training with synthetic data to improve model robustness against adversarial examples.
• Combine SPCAGAN-generated data with a hybrid Bayesian Neural Network (BNN) for end-to-end anomaly detection.
• Employ visualization techniques (KDE, PCA, t-SNE) to evaluate the similarity between real and synthetic data distributions.

Figure 1: Multi-stage Process flow in the Proposed Approach

Experimental results

Research questions

• RQ1 Can a GAN-based data augmentation method effectively reduce class imbalance in insider threat detection without relying on real data?
• RQ2 How does the integration of linear manifold learning improve the quality and diversity of synthetic insider threat data?
• RQ3 To what extent does SPCAGAN outperform existing GANs in avoiding mode collapse and converging faster?
• RQ4 Does the hybrid BNN model trained on SPCAGAN-generated data achieve better anomaly detection performance than state-of-the-art models?
• RQ5 How well do the synthetic data distributions match the original data distributions in terms of feature space geometry?

Key findings

• SPCAGAN achieved the highest F1-score of 0.668 on the CERT v5.2 dataset, outperforming all baseline methods including CGAN, WGAN-GP, and ACGAN.
• The model demonstrated superior convergence speed and eliminated mode collapse, as confirmed by t-SNE visualizations showing well-separated clusters for minority insider activities.
• SPCAGAN-generated synthetic data showed high distributional similarity to real data, as evidenced by kernel density estimation and PCA/t-SNE visualizations across multiple features.
• The hybrid BNN model trained on SPCAGAN data achieved a macro F1-score of 0.917 on CERT v5.2, indicating strong performance on minority classes.
• Adversarial training with SPCAGAN-generated data improved model robustness, reducing vulnerability to adversarial perturbations.
• The proposed method achieved a 3.868 F1-score on the CERT v5.2 dataset, significantly outperforming existing methods such as ACGAN (F1=0.621) and WGAN-GP (F1=0.674).

Figure 2: Architecture of CGAN and ACGAN