[Paper Review] Neural Classification of Malicious Scripts: A study with JavaScript and VBScript

Jack W. Stokes, Rakshit Agrawal+1 more|arXiv (Cornell University)|May 15, 2018

Advanced Malware Detection Techniques24 references10 citations

TL;DR

This paper proposes LaMP and CPoLS, deep recurrent neural networks that classify malicious JavaScript and VBScript by processing byte sequences end-to-end using LSTMs and, in CPoLS, 1D convolutions. The best LaMP model achieves a 65.9% true positive rate at 1.0% false positive rate on 296,274 JavaScript files and 69.3% TPR on 240,504 VBScript files, demonstrating high accuracy with minimal input (first 200 characters), enabling fast, scalable detection.

ABSTRACT

Malicious scripts are an important computer infection threat vector. Our analysis reveals that the two most prevalent types of malicious scripts include JavaScript and VBScript. The percentage of detected JavaScript attacks are on the rise. To address these threats, we investigate two deep recurrent models, LaMP (LSTM and Max Pooling) and CPoLS (Convoluted Partitioning of Long Sequences), which process JavaScript and VBScript as byte sequences. Lower layers capture the sequential nature of these byte sequences while higher layers classify the resulting embedding as malicious or benign. Unlike previously proposed solutions, our models are trained in an end-to-end fashion allowing discriminative training even for the sequential processing layers. Evaluating these models on a large corpus of 296,274 JavaScript files indicates that the best performing LaMP model has a 65.9% true positive rate (TPR) at a false positive rate (FPR) of 1.0%. Similarly, the best CPoLS model has a TPR of 45.3% at an FPR of 1.0%. LaMP and CPoLS yield a TPR of 69.3% and 67.9%, respectively, at an FPR of 1.0% on a collection of 240,504 VBScript files.

Motivation & Objective

• To address the growing threat of malicious JavaScript and VBScript, which are prevalent in spearphishing and ransomware attacks.
• To overcome limitations of prior static and dynamic analysis methods that fail to detect obfuscated or unpacked scripts.
• To develop end-to-end trainable deep learning models that learn sequential representations directly from raw script byte sequences.
• To evaluate model performance on large-scale, real-world corpora of benign and malicious scripts with minimal input (first 200 characters).
• To compare the effectiveness of two novel architectures—LaMP (LSTM and max pooling) and CPoLS (convoluted partitioning of long sequences)—for script malware detection.

Proposed method

• The models process raw script files as byte sequences, treating each byte as a token in a sequential input.
• LaMP uses a two-stage architecture: first, one or more LSTM layers learn sequential representations from the byte sequence; second, a dense neural network classifies the final embedding as malicious or benign.
• CPoLS extends LaMP by adding a 1D convolutional layer before the LSTM to partition and extract features from long script sequences, improving handling of longer files.
• Both models are trained end-to-end using supervised learning, allowing joint optimization of the language modeling and classification components.
• The models are evaluated on a large corpus of 296,274 JavaScript and 240,504 VBScript files, with labels derived from sandboxed execution and AV engine analysis.
• Model performance is measured using true positive rate (TPR) and false positive rate (FPR), with results reported at a 1.0% FPR threshold.

Experimental results

Research questions

• RQ1 Can end-to-end trained deep recurrent models effectively detect malicious JavaScript and VBScript from raw byte sequences?
• RQ2 How do the LaMP and CPoLS architectures compare in performance on large-scale, real-world script corpora?
• RQ3 To what extent can models achieve high detection rates using only the first 200 characters of a script?
• RQ4 Can these models outperform traditional static and dynamic analysis methods that rely on feature engineering or API extraction?
• RQ5 Do the learned embeddings from LSTMs and 1D convolutions capture discriminative patterns indicative of malicious behavior?

Key findings

• The best-performing LaMP model achieves a 65.9% true positive rate at a 1.0% false positive rate on 296,274 JavaScript files.
• The best CPoLS model achieves a 45.3% true positive rate at a 1.0% false positive rate on the same JavaScript corpus.
• On VBScript files, the LaMP model achieves a 69.3% true positive rate at 1.0% false positive rate, outperforming CPoLS and competing models.
• The CPoLS model achieves a 67.9% true positive rate at 1.0% false positive rate on VBScript files, demonstrating strong performance with long-sequence modeling.
• Both models achieve high detection accuracy using only the first 200 characters of each script, enabling fast, scalable deployment in production environments.
• The end-to-end training of LaMP and CPoLS allows for discriminative learning across all layers, improving representation quality compared to frozen or pre-trained language models.