[Paper Review] Deep Learning for Unsupervised Insider Threat Detection in Structured Cybersecurity Data Streams

Aaron Tuor, Samuel Kaplan+3 more|arXiv (Cornell University)|Oct 2, 2017

Network Security and Intrusion DetectionComputer Science184 citations

TL;DR

Online unsupervised deep learning detects anomalous user activity in streaming system logs; per-user DNN/LSTM models with interpretable anomaly contributions outperform PCA, SVM, and Isolation Forest baselines on CERT v6.2 data.

ABSTRACT

Analysis of an organization's computer network activity is a key component of early detection and mitigation of insider threat, a growing concern for many organizations. Raw system logs are a prototypical example of streaming data that can quickly scale beyond the cognitive power of a human analyst. As a prospective filter for the human analyst, we present an online unsupervised deep learning approach to detect anomalous network activity from system logs in real time. Our models decompose anomaly scores into the contributions of individual user behavior features for increased interpretability to aid analysts reviewing potential cases of insider threat. Using the CERT Insider Threat Dataset v6.2 and threat detection recall as our performance metric, our novel deep and recurrent neural network models outperform Principal Component Analysis, Support Vector Machine and Isolation Forest based anomaly detection baselines. For our best model, the events labeled as insider threat activity in our dataset had an average anomaly score in the 95.53 percentile, demonstrating our approach's potential to greatly reduce analyst workloads.

Motivation & Objective

• Motivate early insider threat detection from high-velocity, heterogeneous system logs.
• Develop an online unsupervised deep learning system that models normal user behavior to flag anomalies.
• Provide interpretable anomaly explanations by decomposing scores into contributing features.
• Evaluate online DNN and LSTM architectures against standard baselines on the CERT Insider Threat Dataset v6.2.

Proposed method

• Represent system logs as daily, per-user 414-dimensional feature vectors (408 counts + 6 categorical attributes).
• Implement two neural architectures: a Deep Neural Network (DNN) and a Recurrent Neural Network (LSTM) sharing parameters but with per-user state for online learning.
• Predict the next daily feature vector or reconstruct the current one, using probabilistic models to compute anomaly as negative log-likelihood.
• Model the joint distribution over continuous counts and six categorical variables with a factorized approximation and softmax outputs for categoricals.
• Decompose anomaly scores into contributions from individual features to aid analyst interpretation.
• Train online by updating weights with each new user-event vector, maintaining per-user hidden/cell states while sharing global parameters.

Experimental results

Research questions

• RQ1 Can online DNN and LSTM models detect insider threat in streaming, structured cybersecurity data more effectively than traditional baselines?
• RQ2 Does modeling per-user behavior with online training improve anomaly detection and interpretability in real time?
• RQ3 What is the impact of including categorical features versus using only count features on detection performance?
• RQ4 Which prediction target (same time step vs next time step) yields better anomaly detection?
• RQ5 Does a diagonal covariance model for continuous features improve performance over identity covariance?

Key findings

• DNN-Diag and LSTM-Diag substantially outperform PCA, SVM, and Isolation Forest baselines on cumulative recall metrics (CR-400 and CR-1000).
• Including categorical features yields a modest performance gain for some setups but generally not when using count features alone; the count-only models performed best in many experiments.
• Prediction using same time step (reconstruct current input) outperformed next time step in both DNN and LSTM variants.
• Diagonal covariance for continuous features improves performance over identity covariance, especially with online context.
• On the development/test split, certain models achieved 100% recall at daily budgets around 425, with notable recall at smaller budgets (e.g., 250 for 90% recall).
• The best results on the main table show CR-400 = 11.6 and CR-1000 = 35.6 for LSTM-Diag, and CR-400 = 9.2 and CR-1000 = 32.3 for LSTM-Diag-Cat.